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Abstract 

Grossberg’s  Motion  Oriented  Contrast  Filter(MOC)  was  extensively  analyzed(7).  The  output 
from  the  filter’s  “global  motion”  neuronal  layer  was  compared  to  a  noncausal  post-processing  filter 
developed  by  AFIT.  Both  filters  were  shown  to  incorporate  a  weighted,  noncausal  temporal  range 
of  input  data  in  processed  output.  The  global  motion  framework  was  then  implemented  using  a 
physiologically  motivated  pulsed  neural  model  -  the  Pulse  Coupled  Neural  Network(PCNN).  By 
incorporating  both  spatial  and  temporal  data,  the  PCNN  was  shown  to  exhibit  a  common  visual 
illusion,  apparent  motion.  The  existence  of  a  physiological  temporal  processing  range  was  further 
investigated  through  implementation  of  two  multi-modal  experiments  which  integrated  visual  and 
auditory  stimulus  input  channels.  Results  from  the  first  experiment  reinforce  earlier  findings  from 
literature  of  a  temporal  window  for  perception  of  simultaneous  activity.  (Events  occurring  within 
this  window  are  considered  simultaneous;  events  which  span  more  than  one  window  are  considered 
temporally  separate.)  Data  collected  from  the  second  experiment  suggests  future  inputs  from  an 
accessory  auditory  stimulus  impact  current  perception  of  a  visual  stimulus.  The  influence  of  the 
auditory  accessory  stimulus  decreases  as  the  temporal  delay  between  visual  and  auditory  stimulus 
presentation  is  increased  up  to  a  maximum  value  of  approximately  40  milliseconds.  These  tests 
results  suggest  the  existence  of  perceptual  noncausality  in  the  mind  -  awareness  as  a  function  of 
past,  current,  and  future  perceptual  inputs. 
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/.  Introduction 

1.1  Background 

Apparent  motion  -  the  illusion  of  continuous  motion  when  presented  with  a  series  of  still 
frames,  is  well  known  among  neuro-scientists,  who  seek  to  understand  its  origins,  as  well  as  Holly¬ 
wood  directors,  who  use  its  effects  to  entertain  audiences(8).  Due  to  recent  advances  in  the  study 
of  human  cognition,  an  increased  interest  in  the  study  of  apparent  motion  has  emerged.  If  two 
spatially  translated  dots  on  an  otherwise  blank  screen  are  flashed  in  temporal  succession,  individ¬ 
uals  observing  the  dots  will  experience  the  sensation  of  apparent  motion  if  l)the  frequency  of  the 
flashing  dots  is  sufficiently  high,  and  2)the  distance  between  the  two  dots  is  sufficiently  small(8). 
Such  an  observance  is  significant  for  the  following  reasons:  Individuals  perceiving  apparent  motion 
see  one  dot  move  from  the  point  of  the  first  flashing  dot  to  the  point  of  the  second  flashing  dot 
(figure  1).  Since,  in  reality,  no  dots  were  flashed  in  the  time  between  the  first  dot  turn  off  and  the 
second  dot  turn  on,  the  mind  had  to  fabricate  the  perception  of  a  dot  (or  series  of  dots)  in  time 
and  space  (between  the  dots)  based  on  both  future  (the  second  dot)  and  past  (the  first  dot)  inputs 
through  the  visual  system.  As  a  result,  the  mind  appears  to  work  in  a  noncausal  fashion;  that  is, 
it  seems  to  produce  outputs,  the  perception  of  the  in-between  dots,  which  are  not  only  the  result 
of  previous  inputs  (the  first  dot),  but  future  inputs  as  well  (the  second  dot). 

In  a  related  visual  illusion,  a  brief  flash  of  colored  light  followed  immediately  by  a  second  flash 
of  a  different  color  at  the  same  location  may  result  in  the  perception  of  a  single  flash  of  a  third  color 
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Figure  1.  Apparent  Motion(shaded  area  corresponds  to  stimulus  on  at  specific  space-time 
location) 


(5) (figure  2).  The  perception  of  this  single  “apparent  color”  is  affected  by  l)the  intensity  of  each 
color  presented;  2)the  relative  presentation  location  for  each  color;  and  3)the  temporal  length  of 
each  flash.  Once  again,  the  mind  appears  to  fabricate  an  output  based  upon  a  temporal  sequence 
of  inputs.  More  importantly,  perhaps,  is  the  apparent  manipulation  of  time.  The  brain  somehow 
receives  two  separate  temporal  events  and  squeezes  their  perceived  occurrence  into  one  solitary 
perceived  event.  In  effect,  neurological  processing  appears  to  incorporate  a  temporal  processing 
window  to  determine  the  inputs  to  “current”  awareness.  Incorporation  of  such  a  window  allows  the 
brain  to  preserve  and  maintain  a  stable  and  consistent  internal  representation,  or  “world  model,” 
of  an  external  environment. 
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Figure  2.  Apparent  Color 


Taken  alone,  the  occurrences  of  apparent  motion  and  color  appear  to  be  little  more  than 
neurological  slights  of  hand.  However,  an  understanding  of  why  and  how  the  illusions  occur  may 


2 


provide  the  basis  for  a  model  of  higher  brain  activity.  Specifically,  an  understanding  of  how  the 
brain  integrates  the  spatial  and  temporal  aspects  of  data  inputs  might  one  day  lead  to  a  rudimentary 
model  for  consciousness,  the  Holy  Grail  of  modern  neuroscience.  The  potential  benefits  to  be  gained 
from  the  integration  of  an  artificial  consciousness  into  existing  technologies  provide  the  impetus 
behind  the  involvement  of  the  USAF  in  the  search  for  a  solution  to  the  problem  of  modeling  apparent 
motion  and  color.  If  such  a  solution  is  found,  then  the  basis  for  much  higher  level  artificial  thought 
may  be  formed,  thereby  giving  new  meaning  to  the  moniker  “smart  machines”. 

1.2  Problem  Statement  and  Scope 

The  temporal  aspects  of  visual  illusions  will  be  investigated  to  gain  insight  into  a  general 
understanding  of  the  origins  of  conscious  processing.  Analysis  will  be  concentrated  around  l)inves- 
tigation  of  temporally-weighted  processing  filters  developed  separately  by  Dr.  Stephen  Grossberg 
at  Boston  University  ((7)(12)(13)(11))  and  AFIT  (27);  2)simulation  of  this  temporal  filter  concept 
using  a  physiologically-motivated  Pulse  Coupled  Neural  Network;  and  3) implementation  of  a  series 
of  multi-modal  physiological  experiments  to  quantify  the  existence  of  a  sliding  temporal  window  in 
perceptual  processing. 

1.3  Thesis  Organization 

The  following  chapter  will  introduce  the  underlying  concepts  of  Dr  Stephen  Grossberg’s  Mo¬ 
tion  Oriented  Contrast  filter(MOC)  as  they  relate  to  the  temporal  processing  of  visual  illusions(7)(12)(13)(ll). 
The  foundation  for  the  filter,  the  presence  of  a  “Global  Motion  Layer,”  will  be  compared  to  the 
sliding  non-causal  temporal  window  filter  developed  by  AFIT  (27)  to  post-process  pristine  and 
FLIR  moving  images.  In  Chapter  III,  a  pulse  based  neural  network  architecture  will  be  introduced 
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and  used  to  simulate  the  temporal  processing  properties  of  a  Global  Motion  layer.  The  ability 
of  such  a  network  to  reproduce  the  apparent  motion  phenomenon  will  also  be  analyzed.  Chapter 
IV  describes  the  physiological  motivation  and  implementation  of  two  multi-modal  time-dependent 
experiments.  The  first  experiment  demonstrates  the  existence  of  a  perceptional  window  for  “cur¬ 
rent  time.”  The  second  experiment  provides  evidence  that  event  occurence  is  temporally  weighted 
such  that  past  and  future  events  may  affect  perception  of  a  current  event.  Finally,  Chapter  V  will 
discuss  the  conclusions  which  can  be  drawn  from  this  work. 
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II.  Noncausality  in  the  Perception  of  Motion 


2.1  Background 


In  1992,  AFIT  developed  a  neural  network  retinal  model  to  segment  motion  using  spatio- 
temporal  inputs  (27).  The  AFIT  model,  heretofore  referred  to  as  the  Adaptive  Retinal  Model,  used 
as  its  basis  a  model  of  the  fly’s  visual  system  developed  by  Ogmen  and  Gagne  and  added  to  this 
model  two-dimensional  adaptive  input  nodes  (25)  (27).  In  the  course  of  the  research,  AFIT  found 
that  output  data  obtained  from  the  model  clearly  demonstrated  motion  in  one  dimension;  but  this 
ability  was  severely  diminished  when  the  model  was  extended  to  two  dimensions.  (The  additional 
dimension  required  that  output  be  represented  as  a  series  of  images  -  the  result  being  a  loss  of  the 
features  in  the  response  among  the  256  gray  levels  used  to  represent  the  cell  activity  level  (27)). 
In  order  to  focus  on  the  transient  behavior  of  the  input,  (and  thereby  minimize  the  model’s  loss 
in  performance),  two  post-processing  techniques  were  developed:  moving  average  subtraction  and 
thresholding. 

AFIT  implemented  four  versions  of  the  moving  average  subtraction  techniques.  They  included 
the  causal  and  non-causal  level-window  average  and  a  causal  and  non-causal  gaussian  window 
average.  An  illustration  of  these  moving  averages  is  shown  in  figure  36(27).  The  nine  images  shown 
represent  nine  frames  of  output  spanning  a  “window”  in  time  centered  on  the  image  currently  being 
processed.  The  moving  average  subtraction  methods  were  described  as  follows: 

The  moving  average  process  calculates  an  average  for  each  pixel  and  subtracts  the 
current  pixel  value.  In  the  causal  moving  average,  the  averaging  window  extends  only 
into  the  past.  No  future  data  is  needed  in  this  process  and  the  window  can  be  level- 
windowed  or  gaussian  in  shape.  The  noncausal  moving  average  uses  either  window  type 
but  adds  as  many  future  images  as  past  images.  This  scheme  requires  the  model  to 
delay  its  output  until  it  can  calculate  the  current  frame  value  (27). 
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Figure  3.  Adaptive  Retinal  Model  Post-Processing  (Moving  Averages) 

AFIT  applied  both  techniques  to  the  output  generated  from  a  one-dimensional  simulation  in  which  a 
large  moving  rectangle  is  passed  over  a  smaller  stationary  rectangle.  The  input  and  output  from  the 
adaptive  retinal  model,  prior  to  post  processing,  are  shown  in  figures  4  (27)  and  5  (27),  respectively. 
Notice  the  ridge  and  dip  in  the  forward  wave  when  the  two  rectangles  cross.  The  effects  of  the 


Input  V^lue 


Figure  4.  Adaptive  Retinal  Model  Input 


causal,  level- window  moving  average  on  the  retinal  model  output  can  be  seen  in  figure  6  (27).  As 
expected,  AFIT  observed  a  severely  diminished  response  in  the  plateaus,  mesas,  and  rippling  of  the 
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Figure  5.  Adaptive  Retinal  Model  Output 


output.  Unfortunately,  while  the  overall  response  was  decreased,  the  initial  transient,  forward,  and 
trailing  waves  were  enhanced.  In  contrast,  the  output  from  the  non-causal,  level  window  moving 
average  (shown  in  figure  7  (27))  provided  a  much  clearer  indication  of  motion.  Specifically,  the 
trailing  wave  was  inverted  -  resulting  in  a  distinguishable  plateau  between  the  forward  and  trailing 
waves.  Outside  of  the  motion  region,  the  background  plane  remained  nearly  plane.  When  the 
second  post  processing  technique,  thresholding,  was  applied  to  this  output,  the  leading  and  trailing 
edges  of  the  moving  rectangle  were  clearly  shown  (figure  8  (27)).  The  post-processing  described 
above  referred  only  to  causal  and  noncausal  level-windowed  versions.  AFIT  later  applied  all  four 
versions  of  the  moving  average  window  to  two  2D  image  sequences:  a  computer-generated  pristine 
image  sequence,  and  a  real  Forward  Looking  Infrared  Radar(FLIR)  image  sequence.  Examples 
of  both  image  sequences  are  represented  in  figures  9(27)  and  10(27),  respectively.  (Figure  9  also 


Figure  6. 


Post  Processing:  Causal,  Level- Window  Average  (transient,  leading,  and  trailing  edges 
shown) 
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Figure  7.  Post  Processing:  Non-Causal,  Level-Window  Average  (transient,  leading,  and  inverted 
trailing  edges  shown) 


Figure  8.  Post  Processing:  Thresholding  (leading  and  trailing  edges  of  moving  rectangle  shown) 
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includes  example  sequences  for  the  image  input  and  O&G  model  output  for  sake  of  comparison.) 
Using  the  same  post-processing  techniques  described  earlier,  AFIT  concluded  that  the  non-causal, 
gaussian-windowed,  moving  average  produced  the  best  response  for  both  the  computer  generated 
image  sequence,  as  well  as  the  FLIR  image  sequence.  Specifically,  AFIT  observed;  “The  causal 
filter  shows  gradients  of  the  output  imagery  based  only  on  past  information,  while  the  noncausal 
filter  adds  future  data  to  give  direction  information(emphasis  added)(27).”  From  a  performance 
standpoint,  AFIT  demonstrated  that  the  integration  of  a  non-causal  window  greatly  enhances  the 
calculation  of  form  and  motion  in  time. 


Input  O&G  Response  Adaptive  Response 


Frame  15  of  Input  Sequence 

Figure  9.  Computer  Generated  2D  Pristine  Image  Frames(input,  two-dimensional  O&G  model 
output,  and  AFIT  Adaptive  model  output  shown) 
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Figure  10.  Forward  Looking  Infrared  Radar(FLIR)  Image  Frames  (output  from  noncausal  moving 
average  post-processing  of  Adaptive  Model) 

2.2  Noncausality  in  Grossberg’s  Oriented  Contrast  Filter(MOC) 

Although  never  referred  to  as  such,  Stephen  Grossberg  implements  a  perceptual  non-causal 
filter  as  part  of  the  level  5  processing  of  his  Motion  Oriented  Contrast  Filter.  Through  implementa¬ 
tion  of  spatially  broad  receptive  fields,  level  5  cells  receive  inputs  from  localized  groups  of  lower  level 
cells.  These  inputs  precede  “perception”  of  visual  activity  at  corresponding  retinal  locations  and 
can  be  described  as  “perceptually  non-causal.”  Before  a  discussion  of  these  non-causal  properties 
can  proceed,  however,  a  brief  graphical  review  of  the  filter’s  first  four  levels  is  presented. 

2.2.1  Generating  Motion  Signals  at  Level  4  Neurons.  Grossberg  originally  developed 
his  Motion  Oriented  Contrast  Filter (MOC)  because  the  previous  version,  the  Oriented  Contrast 
filter(OC),  was  insensitive  to  direction  of  motion(7).  To  produce  the  necessary  motion  sensitivity, 
the  retinal  input  (level  1)  is  used  as  inputs  by  both  sustained  cells  (level  2)  and  transient  cells 
(level  3)  to  produce  sustained  and  transient  responses  at  each  neuron  “position”.  One  way  to 
visualize  such  an  arrangement  is  shown  in  Figures  11  and  12.  Figure  11  represents  the  spatial 
relationship  between  level  2  sustained  cells  and  level  1  input  responses.  For  the  sake  of  clarity,  the 


Figure  11. 


Figure  12. 


I  Level  2 
>  Bright-Dark 
I  Cells 


Level  1  Cells 


Interaction  Between  level  1  cells(retinal  photo-receptors)  and  Bright-Dark  level  2  (sus¬ 
tained)  cells 


Interaction  Between  level  1  cells(retinal  photo-receptors)  and  Transient-On  level  3 
cells  (with  space-time  receptive  fields) 
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relationship  is  shown  in  one  dimension  only.  In  addition,  the  level  2  cells  shown  represent  only 
those  cells  sensitive  to  spatial  Bright-Dark  changes  in  luminance.  To  fully  represent  level  2  cells, 
an  additional  set  of  cells  sensitive  to  spatial  Dark-Bright  changes  in  luminance  must  be  included. 
A  Bright-Dark  Level  2  cell  shown  in  Figure  11  will  produce  a  response  if  its  respective  level  1 
cells  produce  excitatory  inputs  for  the  “Bright”  side  of  the  level  2  cell  and  inhibitory  inputs  for 
its  “Dark”  side(12).  Dark-Bright  level  2  cells  respond  under  complementary  brightness  conditions. 
Since  the  illustration  shown  is  limited  to  one  dimension,  the  result  of  such  a  setup  is  a  network 
responsive  to  relative  changes  in  luminance  among  neighboring  level  1  cells(7). 

Figure  12  represents  the  temporal  relationship  between  level  3  transient  cells  and  level  1  input 
responses.  Level  3  transient  on-cells  (shown  in  the  illustration)  respond  if  the  level  1  cell  at  time 
t  -I- 1  is  greater  than  the  activity  of  that  cell  at  time  t.  Similarly,  transient  off-cells  respond  if  the 
activity  of  their  corresponding  level  1  cells  decrease  over  time.  The  level  3  representation  shown  in 
figure  12  is  greatly  simplified  for  ease  of  explanation.  For  an  in-depth  explanation  of  the  transient 
stages  required  for  accurate  representation,  refer  to  Stephen  Grossberg’s  complete  presentation  of 
the  MOC  filter(reference(7)). 

At  this  point,  level  2  and  level  3  cells  contain  information  as  to  the  potential  direction  of 
contrast,  as  well  as  transient  behavior  of  the  response.  By  gating  the  level  2  and  3  outputs  with 
each  other,  a  level  4  “local  motion”  signal  is  created(7).  A  representation  of  this  gating  is  shown 
in  figure  13. 

2.2.2  Noncausality  in  the  MOC:  Calculating  Level  5  Cell  Inputs.  As  was  previously 
stated,  Grossberg’s  goal  in  designing  the  Motion  Oriented  Contrast  Filter  was  to  produce  a  filter 
whose  outputs  would  be  motion  signals  which  could  be  competed  in  a  competition-cooperation  loop 


13 


Figure  13.  Gating  of  Level  2  Sustained  Cells  with  Level  3  Transient  Cells 

analogous  to  the  original  Boundary  Contour  System(12).  The  cells  are  “global”  in  the  sense  that 
the  spatial  “footprint”  (ie.,  receptive  field)  of  level  5  cell  inputs  spread  out  across  space.  For  the  one 
dimensional  view  shown  in  figure  14,  each  level  5  cell  represents  a  “global”  position  and  receives 
inputs  from  level  4  cells  across  space  at  each  time  increment.  If  this  example  is  expanded  to  two 
dimensions,  the  spatial  footprints  appear  as  a  series  of  overlapping  concentric  circles  corresponding 
to  each  global  motion  cell  represented  in  the  field  (figure  15).  The  maximum  “reach”  of  the  footprint 
is  dependent  upon  the  inter-connectivity  of  the  level  5  and  level  4  cells.  For  the  examples  shown, 
the  receptive  field  spans  across  5  level  4  cells.  The  output  of  each  level  5  cell  is  dependent  upon 


Figure  14.  1-Dimensional  Representation  of  Level  5  Cells  Receptive  Fields 
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Level  S  Cell  (receptive  field  -  5) 


Level  4  Cell  Grid 


Figure  15.  2-Dimensional  Representation  of  Level  5  Cells  Receptive  Fields 


a  summation  of  weighted  inputs  from  the  connected  level  4  cells.  Both  Grossberg  and  AFIT  use  a 
Gaussian  kernel  to  generate  input  weights.  For  Grossberg’s  MOC,  this  kernel  is  spatial;  at  individual 
time  increments,  inputs  from  different  level  4  cells  are  multiplied  by  a  distance-dependent  Gaussian 
weight,  and  then  summed  together  for  input  into  a  level  5  cell(7).  In  the  previous  AFIT  research, 
the  Gaussian  profile  was  used  in  a  temporal  fashion.  For  each  spatial  position,  the  contribution  of 
each  temporal  response  is  calculated  by  first  scaling  the  response  by  a  time-dependent  Gaussian 
weight,  then  averaging  the  result  obtained  with  other  corresponding  temporal  responses  at  the  same 
location(27).  The  weighting  profile  used  here  is  crucial  -  the  use  of  a  linear  or  constant  profile  will 
not  produce  satisfactory  results.  The  level  5  output  generated  using  the  weighted  level  4  inputs 
is  competed  along  with  all  other  level  5  cells  corresponding  to  the  same  motion  direction  in  what 
Grossberg  describes  as  a  “winner  take  all”  network(7).  The  result,  then,  is  the  excitation  of  one 
level  5  motion  cell  and  inhibitation  of  those  cells  surrounding  it. 

Given  the  level  5  configuration  described  above,  how  does  this  model  account  for  the  non¬ 
causality  of  AFIT’s  post-processing  model  and,  as  a  by-product,  produce  apparent  motion?  The 
answer  to  this  question  lies  in  the  temporally-dependent  cell  activity  level.  Recall  that  the  activity 
level  (defined  here  as  pulses/unit  time)  of  a  given  cell  does  not  rise  or  fall  instantaneously;  rather. 
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the  activity  level  changes  over  time  depending  on  both  the  current  value  of  the  level  as  well  as 
inputs  from  other  neurons.  In  addition,  this  cell  activity  level  must  cross  a  given  threshold  before 
producing  an  output.  As  a  result,  a  temporal  delay,  deltat,  exists  between  the  onset  of  a  stimulus 
and  the  time  that  a  given  neuron  “acknowledges”  the  stimulus  input  by  outputting  a  pulse  (figure 
16).  For  purposes  of  this  discussion,  the  crossing  of  the  internal  cell  threshold  will  designate 
“perception”  for  higher-order  processing.  With  this  definition  in  mind,  two  separate  time  axis  now 
exist:  real  time  and  “perceived”  time.  Real  time  denotes  passage  of  time  external  to  the  brain;  ie., 
the  time  when  an  event  occurs.  The  rise  and  fall  of  the  internal  cell  activity  level  is  a  function  of 
real  time  -  inputs  to  the  level  5  cell  contribute  to  the  internal  cell  activity  level  as  they  enter  the 
synapse.  In  contrast,  perceived  time  refers  to  the  passage  of  time  experienced  or  acknowledged  by 
the  brain.  Until  the  occurence  of  an  event  produces  a  cell  activity  level  above  the  internal  perceptual 
thresholds  described  earlier,  no  awareness  of  that  event  is  experienced  in  the  brain.  As  a  result, 
the  perception  of  a  stimulus  is  a  function  of  real  time,  tr,  plus  some  internal  event  processing  time, 
deltat,  which  may  vary  depending  upon  the  type  and  number  of  feeding  inputs.  Since  passage 
of  internal  perceptual  time  is  referenced  to  awareness  of  a  stimulus,  the  deltat  processing  time  is 
nonexistent  on  the  perceptual  axis. 

For  a  unitary  input  at  level  5,  (that  is,  only  one  level  4  neuron  firing  during  an  arbitrary 
window  in  real  time),  the  level  5  cell  will  fire  upon  reaching  its  internal  threshold.  Here,  the 
perception  threshold  is  only  dependent  upon  one  level  4  input  so  the  “perception”  of  this  input 
occurs  at  +  deltat .  However  if  this  single  stimulus  is  removed  and  another  stimulus  added 
at  a  later  time  at  a  different  location,  a  different  scenario  arises(figure  17).  The  removal  of  the 
stimulus  does  not  simultaneously  remove  its  effects  upon  the  level  5  cell.  Rather,  cell  activity  in 
level  5  will  begin  to  decay  over  time.  As  it  does  so,  the  activity  may  fall  below  the  “perception” 
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Figure  16.  Representation  of  temporal  delay,  deltat  between  stimulus  onset  and  perceptual  onset 

threshold  -  thereby  preventing  output.  Note:  the  activity  level  may  not  be  zero;  it  may  simply 
be  below  the  perception  threshold.  In  a  similar  manner,  cell  activity  from  a  level  4  input  cannot 
instantaneously  force  a  level  5  cell  to  output,  that  is,  produce  a  signal  which  is  “perceived”  by 
higher  processing.  As  was  the  case  for  the  previous  stimulus  turning  off,  a  new  stimulus  turning 
on  causes  a  level  4  cell  to  increase  its  affect  upon  a  level  5  cell  over  time.  Once  again,  this  input 
may  or  may  not  enable  the  level  5  cell  to  reach  threshold.  What  happens,  then,  if  the  circumstance 
arises  whereby  a  stimulus  turning  on  and  another  turning  off  occur  in  such  a  way  as  to  produce 
increasing  and  decreasing  cell  activity  levels  which  are  simultaneously  fed  into  a  level  5  cell?  The 
answer  is  perceptual  noncausality,  and,  if  the  conditions  are  “right”,  apparent  motion.  Until  a  level 
5  cell  exceeds  its  threshold,  nothing  is  perceived  to  occur.  Similarly,  perception  of  a  stimulus  is 
maintained  as  long  as  the  threshold  is  exceeded.  Below  this  threshold,  however,  remnants  of  both 
“old”  stimuli  and  new  “stimuli”  may  (after  undergoing  gaussian  spatial  filtering)  add  together  to 
produce  a  result  which  is  above  the  perception  threshold  and,  therefore,  perceived.  This  result  is 
crucial:  since  some  of  the  stimuli  are  “old”  in  the  sense  that  they  occurred  in  the  past,  they  are 
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Figure  17.  Temporal  Overlap  of  2  Stimulus  Inputs  at  Level  5 

now  past  their  maximum  cell  activity  level  so  any  perception  associated  with  these  stimuli  may 
have  already  occurred.  In  a  similar  fashion,  level  4  cells  responding  to  a  new  stimuli  have  not 
reached  their  full  activity  level  and  may  not  yet  be  perceived  at  level  5.  Conceivably,  the  result 
of  the  scenario  described  above  could  be  the  following:  at  some  time  a  stimulus  response  is 
perceived  at  a  level  5  cell(this  is  the  first  actual  stimulus  response);  after  some  perceptual  time 
tp2)  a  second  stimulus  response  occurs  at  a  different  level  5  cell  location  where  no  actual  stimulus 
occurred(this  response  is  apparent  -  no  stimulus  initiated  it);  finally,  the  last  stimulus  response 
occurs  some  perceived  tp^  later  at  yet  another  level  5  cell  location(once  again,  this  is  an  actual 
response  to  a  real  stimulus).  The  combination  of  rising  and  falling  activity  levels  produces  a  false 
response  which,  when  perceived,  appears  to  be  a  function  of  the  earlier  and  later  signals.  Based  on 
perception,  the  result  is  noncausal.  Simulation  of  apparent  motion  using  the  MOC  filter  is  shown 
in  Figure  18.  For  additional  examples  of  MOC  output,  see  appendix  B. 

The  interaction  of  rising  and  falling  level  4  cell  activity  level  inputs  at  level  5  cells  can  be 
tied  back  to  AFIT’s  non-causal  window  (see  Figure  36)  by  viewing  the  contribution  of  individual 
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Figure  18.  Simulation  of  Apparent  Motion  using  Grossberg’s  MOC  Filter 

cell  activity  levels  which  are  summed  together  at  level  5  cells.  A  stimulus  analogous  to  AFIT’s 
time  frame  t_4  has  a  negligable  effect  upon  the  output  perceived  at  frame  t;  the  cell  activity 
generated  by  the  earlier  frame  is  approaching  zero  by  the  time  the  snapshot  at  time  t  is  taken,  so 
its  contribution/effect  upon  stimulus  perception  at  time  t  is  negligable.  Similarly,  the  cell  activity 
level  of  the  stimulus  which  will  be  perceived  at  frame  t+4  is  still  small  at  perceived  “current”  time 
t,  so  its  contribution  to  level  5  cell  activity  (and  therefore,  perception)  at  this  time  is  small. 

To  demonstrate  the  similarity  between  the  Grossberg  and  AFIT  models,  a  series  of  stimuli 
profiles  are  input  into  a  MOC  filter.  The  stimuli,  shown  in  figure  19,  are  each  presented  for  160 
ms  (simulated),  and  are  separated  by  gaps  of  210,  170,  130,  90,  50,  and  10  ms.  For  purposes  of 
comparison  the  midpoint  between  each  stimulus’  offset  is  set  equal  to  to,  current  time.  (This  is  the 
same  format  used  to  explain  the  AFIT  model  in  figure  36).  Using  this  time  frame,  the  offsets  for 
each  stimulus  profile  occur  at  t±i8o,  t±i6o,  t±i4o,  t±i2o,  ±i00)  and  t±80  ms,  respectively. 

The  MOC  output  at  to  for  each  of  the  six  input  profiles  is  plotted  in  figure  20.  As  was  the  case 
with  AFIT’s  noncausal  Gaussian  window,  the  effect  of  a  stimulus  input  upon  level  5  cell  activity 
decreases  as  its  temporal  proximity  to  to,  current  perceptual  time,  increases. 
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Figure  19.  MOC  Input  Profiles 


Figure  20.  Effect  of  Temporal  Stimulus  Separation  on  Level  5  Activity  at  t\ 


2.3  Conclusion 


Neither  AFIT  nor  Grossberg  set  out  in  their  respective  research  efforts  to  investigate  the  pres¬ 
ence  of  a  perceived  noncausality  in  motion  processing.  Yet,  as  was  demonstrated  in  the  preceding 
pages,  the  presence  of  a  noncausal  element  provided  a  necessary  link  in  the  understanding  of  visual 
processing  for  both  research  efforts.  In  the  case  of  Dr.  Grossberg,  the  perception  of  noncausality 
forced  any  would-be  explanation  of  apparent  motion  to  account  for  the  binding  of  past,  present, 
and  future  inputs.  For  AFIT,  the  incorporation  of  non-causality  in  retinal  model  processing  was 
due  less  to  physiological  accuracy  than  it  was  to  bottom  line  performance:  AFIT  settled  upon 
a  system  which  worked.  The  work  performed  here  has  served  simply  to  highlight  the  non-causal 
perspective  for  both  Grossberg’s  work,  as  well  as  AFIT’s  Adaptive  Retinal  Model.  In  light  of  the 
fact  that  researchers,  led  by  the  likes  of  Dr.  Benjamin  Libet,  have  begun  to  uncover  new  evidence 
as  to  the  existence  of  subjective  temporal  processing  within  the  brain,  it  should  come  as  no  surprise 
that  the  best  model  for  human  visual  processing  is...  human  visual  processing(22). 
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III.  Simulation  of  Apparent  Motion  in  a  Pulse  Coupled  Neural  Network 
3.1  Background 

As  discussed  previously  in  Chapter  2,  AFIT  developed  an  adaptive  retinal  model  to  segment 
simulated  and  real  FLIR  moving  images  (27).  During  the  course  of  data  post-processing,  optimal 
segmentation  occurred  as  the  result  of  a  non-causal  moving  average  window.  That  is,  the  model 
produced  its  best  output  if  post-processing  utilized  both  past  and  “future”  time  slices  when  cal¬ 
culating  the  current  output  seen  by  the  model.  In  a  related  development.  Dr  Stephen  Grossberg, 
as  part  of  a  continuing  effort  to  develop  a  biological  foundation  for  the  phenomenon  of  apparent 
motion,  introduced  a  Motion  Oriented  Contrast  filter(MOC)  for  use  in  a  motion  Boundary  Contour 
System  (7).  In  general,  the  MOC  filter  extended  Grossberg’s  concept  of  competition/cooperation 
among  orientation  signals  to  local  motion  signals.  Each  local  motion  cell  was  the  result  of  succes¬ 
sive  layers  of  oriented  sustained  and  transient  cells  whose  internal  activity  levels  both  increased 
and  decreased  according  to  the  following  generic  differential  equation: 

flT 

-  =  -Ax  +  {B-  x)I  (1) 

where  A  is  the  passive  decay  rate  parameter,  B  is  the  maximum  activity  level  of  the  cell,  and  I 
is  the  resulting  summation  of  weighted  inputs.  The  resulting  local  motion  signal  responses  were 
then  fed  into  a  grid  of  corresponding  global  motion  cells  (  each  receptive  feeding  field  spanned 
the  spatial  dimensions  of  the  lower  cell  layers  -  hence  the  term  “global”).  The  strength  of  each 
feeding  input  was  determined  by  the  spatial  distance  from  the  global  motion  cell  to  each  lower 
local  motion  cell.  In  what  Grossberg  described  as  a  winner-take-all  competitive  network,  the  global 
motion  cell  possessing  the  maximum  input  population  was  determined  to  be  the  “winner”  (7). 
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Viewed  temporally,  this  competitive  framework  produced  localized  responses  in  the  global  motion 
cell  layer  which  corresponded  to  movement  of  a  stimulus  across  the  cell  grid.  Grossberg  and  Rudd 
demonstrated  that  a  temporal  overlap  of  decaying  and  growing  inputs  from  spatially  separated 
local  motion  cells  could  produce  a  global  motion  cell  with  maximum  response  between  the  two 
stimulus  inputs. 

The  foundation  of  Grossberg’s  MOC  filter,  a  “Global-Motion  Cell  Layer”  whose  inputs  are 
separated  in  both  time  and  space,  is  integral  to  the  formulation  of  a  theory  which  might  be  used  to 
explain  a  wide  range  of  perceptual  illusions.  However,  any  demonstration  of  this  theory  is  contin¬ 
gent  upon  the  underlying  structure  of  the  neurological  unit  used  in  its  implementation.  Throughout 
his  research.  Dr.  Grossberg  demonstrates  MOC  operation  using  a  simplified  neuron  model  whose 
inputs  and  outputs  are  continuous.  Internal  cell  activity  is  a  product  of  an  internal  “leaky  inte¬ 
grator”  whose  inputs  are  continuous  outputs  from  lower  level  cells.  Once  the  internal  cell  activity 
level  exceeds  a  given  threshold,  the  neuron  initiates  an  output  signal  (action  potential)  which  is 
proportional  to  the  aforementioned  cell  activity  level.  Under  this  model,  then,  cell  output  may 
increase  or  decrease  depending  upon  the  state  of  its  inputs. 

Such  a  model  would  prove  adequate  if  the  “global  motion”  layer  described  by  Grossberg  was 
thought  to  exist  in  the  retina.  There,  neurons  in  the  first  three  layers  (cones/rods,  horizontal  cells, 
and  bipolar  cells)  have  been  shown  to  exhibit  a  continuous  output  characteristic(31).  However, 
Grossberg’s  research  points  to  area  MT  (V5)  of  the  visual  cortex  as  the  most  probable  location 
for  the  global  interactions  described  in  his  work(13).  In  this  region  of  the  brain,  research  data  has 
shown  that  neuron  output  is  not  continuous;  rather,  it  is  a  series  of  pulses.  When  the  internal  cell 
activity  level  exceeds  its  respective  threshold,  the  result  is  the  release  of  neurotransmitter  chemicals 
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which  produce  the  characteristic  action  potential  spike  output.  The  action  potential  threshold  is 
a  function  of  the  amount  of  neurotransmitter  present  in  the  synapse.  As  such,  a  “firing”  depletes 
this  resevoir  and  raises  the  activation  potential  threshold.  After  each  firing,  the  synapse  undergoes 
a  refractory  period,  during  which  time  the  higher  activation  threshold  allows  time  for  the  level  of 
neurotransmitters  in  the  synapse  to  be  replenished.  The  net  result  is  a  single  pulse  (if  the  neuron 
inputs  persist  long  enough  such  that  their  weighted  sum  exceeds  the  activation  potential  threshold) 
or  a  pulse  train  (if  the  neuron  inputs  continue  to  contribute  to  the  internal  cell  activity  level  such 
that  the  activation  potential  threshold  is  repeatedly  exceeded  after  successive  synaptic  refractory 
periods). 

In  fact,  research  performed  by  Gray,  Konig,  Engel  and  Singer,  as  well  as  Eckhorn  et  al.  has 
shown  that  neurons  in  the  cat  visual  cortex  produce  oscillatory  responses  in  the  range  40-60  Hz 
(9)  (6).  A  meaningful  examination  of  Grossberg’s  Global  Motion  Layer  theory  requires,  then,  that 
it  be  implemented  using  a  pulse  based  neurological  model.  One  such  model  which  exhibits  this 
characteristic  and,  in  addition,  possesses  the  capability  to  synchronize  ’similar’  pulsed  output,  is 
the  Pulse  Coupled  Neural  Network  (PCNN). 

3.2  PCNN  Description 

The  Pulse  Coupled  Neural  Network  was  conceived  as  a  “..minimal  model  to  explain  the 
experimentally  observed  synchronous  feature  dependent  activity  of  neural  assemblies  over  large 
cortical  distances  in  the  cat  cortex”  (15).  The  model  is  composed  of  two  types  of  functional  synapses: 
feeding  synapses,  junctions  found  in  the  direct  stimulus-driven  signal  path;  and  linking  synapses, 
auxiliary  signals  from  neighboring  neurons  which  modulate  the  feeding  synaptic  inputs  (6).  The 
resulting  modulated  feeding  input  is  fed  into  the  third  main  component  of  the  model  neuron,  the 
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pulse  generator  (or  spike  encoder  (6)).  All  three  components  are  modeled  as  leaky  integrators  (6). 
A  graphical  representation  of  a  generic  PCNN  neuron  is  presented  in  Figure  21(2). 


A  Model  Neuron  in  a  PCNN 


Figure  21.  Pulse  Coupled  Neural  Network 


3.3  Model  Operation 

When  an  input  pulse  is  received  via  a  feeding  or  linking  synaptic  junction,  the  leaky  integrator 
(which  represents  the  internal  activity  level  at  that  junction)  is  charged  such  that  a  rise  in  the  output 
amplitude  is  realized.  Since  the  input  is  a  pulse  and  not  a  continuous  signal,  the  sharp  rise  is  followed 
immediately  by  an  exponential  decay  of  the  signal  (determined  by  a  leakage  time  constant)(6).  The 
sum  of  all  feeding  leaking  integrator  inputs  is  modulated  by  a  combination  of  a  “linking  strength 
factor”  and  the  sum  total  of  all  linking  leaking  integrator  inputs.  The  resulting  product  represents 
the  “membrane  voltage”  or  internal  activity  level  of  the  neuron(6).  This  voltage  is  compared  against 
a  variable  internal  threshold  also  represented  as  a  leaky  integrator.  If  the  membrane  voltage  exceeds 
the  internal  threshold,  an  output  spike  is  formed.  When  this  event  occurs,  the  internal  threshold  is 
recharged  to  an  arbitrarily  high  value  such  that  the  membrane  voltage  cannot  exceed  the  internal 
threshold  during  and  immediately  after  the  pulse  output(6).  During  the  interval  in  which  the 
internal  threshold  remains  above  the  membrane  voltage,  the  neuron  experiences  a  refractory  period 
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-  no  pulse  output  can  be  generated.  Once  the  threshold  has  decayed  to  a  level  at  or  below  the 
membrane  voltage,  the  pulse  output  sequence  is  repeated. 


3.4  Model  Implementation 


Note:  The  equations  which  follow  are  representative  derivations  of  work  performed  by  Eck- 
horn,  Johnson,  Rosenstengel,  and  Broussard.  They  are  provided  here  as  a  minimal  foundation  for 
a  digital  implementation  of  the  Pulse  Coupled  Neural  Network.  For  an  in-depth  analysis  of  the 
equations,  please  refer  to  the  accompanying  references(2)(26). 

The  output  of  a  PCNN  neuron  can  be  represented  as: 


Yj{t)  = 


0 


if  Uj{t)>Oj{t) 
otherwise 


(2) 


(26)  (2). Here,  a  pulse  is  produced  only  when  the  internal  membrane  voltage,  Uj{t),  is  greater  than 
or  equal  to  the  internal  threshold,  Oj{t). 

The  internal  threshold,  Oj{t),  is  modeled  as  a  leaky  integrator  with  time  constant  r  and 
amplitude  gain  V’^(26)  (2),  and  is  a  function  of  a  threshold  offset,  O)’(t),  the  previous  threshold 
value,  Oj{t  —  1),  and  the  neuron  output,  Yj{t): 


Oj(t)  =  OJ(t)  +  Oj(t  -  l)e^  -t-  V%(t)  (3) 

(26)  (2)  The  internal  membrane  voltage,  Uj(t),  is  a  function  of  the  total  feeding  input,  Fj(t), 
modulated  by  the  product  of  the  total  linking  input,  Lj(t)  and  a  linking  strength  coefficient,  /3, 
such  that: 

Uj(t)  =  Fj(t)(l  +  ^Lj(t))  (4) 
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(26) (2)  The  feeding  input,  Fj{t),  is  a  function  of  a  single  (optional)  analog  input,  Ij(t),  and  the 
weighted  sum  of  synaptic  inputs  from  other  neurons(26): 


=  +  (5) 

where  F'kj  (t)  is  the  synapse  between  the  fc*'*  feeding  neuron  and  the  current  neuron. 

Each  feeding  input  synapse,  Fkj(t),  is  modeled  as  a  leaky  integrator  with  time  constant 
and  gain  amplitude: 

F,jit)  =  F,j{t  -  l)e^  +  V^Xj{t)wf,  (6) 

Here,  Xj{t)  is  the  output  of  the  synaptic  input  and  is  the  spatially  determined  Gaussian 
weight. 

Similarly,  the  linking  input,  Lj{t),  is  a  function  of  the  weighted  sum  of  synaptic  linking  inputs 
from  other  neurons: 

Lj{t)  =  J2Lkj{t)  (7) 

where  Lkj{t)  is  the  synapse  between  the  linking  neuron  and  the  current  neuron. 

Each  linking  input  synapse,  Lkj{t),  is  modeled  as  a  leaky  integrator  with  time  constant 
and  gain  amplitude: 

Lkj{t)  =  Lkj{t  -  l)e^  +  V%{t)wf,  (8) 

Here,  Yj{t)  is  the  output  of  the  synaptic  input  of  the  current  neuronal  layer  and  wf^.  is  the 
spatially  determined  Gaussian  weight. 
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3.5  Simulation  of  Neuronal  Activity 

One  of  the  underlying  foundations  for  the  PCNN  network  is  its  ability  to  transform  the  lu- 
minant  intensity  of  a  spatial  representation  into  a  pulse-based  temporal  representation.  To  demon¬ 
strate  this  transformation,  three  point  source  inputs  (figure  22)  are  entered  into  a  PCNN  with  the 
operating  parameters  shown  in  Table  1.  The  first  input,  with  luminance  intensity  I  =  0.2,  is 


Figure  22.  PCNN  Inputs:  a)I=.2  b)I=l  c)I=0 

entered  into  the  network  at  time  0  (iteration  1).  At  iteration  25,  this  stimulus  input  is  removed  and 
replaced  with  the  second  input,  with  a  luminance  intensity  I  =  1.0.  Finally,  at  iteration  50,  the 
second  input  is  removed  so  that  no  stimulus  is  present  for  the  duration  of  the  simulation(input  3) . 
The  results  of  this  ordered  presentation  of  stimuli  upon  a  PCNN  input  layer  are  shown  in  Figure 
23.  The  information  signifying  the  increased  intensity  seen  at  iteration  25  is  conveyed  through 
an  increase  in  the  pulse-rate  output.  Similarly,  the  removal  of  the  same  stimulus  at  iteration  50 
produces  a  sharp  decline  in  pulse-rate,  followed  shortly  thereafter  by  an  absence  of  any  output. 

3.5.1  Multi-Layer  Interactions  in  the  PCNN.  The  capability  to  add  both  feeding  and/or 
linking  inputs  with  an  almost  infinite  number  of  weighting  schemes  to  the  basic  underlying  design 
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Figure  23.  PCNN  Representation  of  Intensity  Variation 
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allows  the  PCNN  model  to  function  as  a  “universal  filter”  in  regards  to  information  processing.  To 
illustrate  this  point,  an  input  pattern  of  uniform  intensity  (shown  in  Figure  24)  is  fed  into  a  2-layer 
PCNN  network.  Initially,  both  the  feeding  and  linking  input  radii  are  set  equal  to  zero  so  that 
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Figure  24.  PCNN  Solid  Input  Pattern 

the  upper  layer  neurons  receive  only  pulsed  outputs  from  the  lower  level  neurons  which  correspond 
to  the  same  spatial  position  in  the  network.  The  result,  as  shown  in  Figure  25,  is  a  (repeated) 
simultaneous  pulse  output  of  those  neurons  with  lower  level  inputs.  The  one  significant  difiFerence 
between  pulsed  outputs  from  each  layer  is  the  inter-layer  pulse  delay,  caused  by  the  presence  of 
a  higher  internal  threshold,  0,  in  the  upper  layer.  This  delay  is  illustrated  in  Figure  26.  If  the 
feeding  radius  at  level  2  is  increased  to  2  while  the  linking  radius  remains  0,  the  pulsed  output 
from  the  lower  layer  no  longer  mirrors  the  lower  input  layer  (Figure  27).  Uniform  pulsed  output  is 
now  replaced  by  a  series  of  pulses  which  reflect  the  degree  of  feeding  inputs  from  lower  level  cells. 
If,  conversely,  the  linking  radius  at  level  2  is  increased  to  2  while  setting  the  feeding  radius  to  0, 
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Figure  27.  PCNN  Level  2  Outputs(feeding  radius  =  2,  linking  radius  =  0) 

pulsed  output  at  layer  2  changes  once  again.  To  illustrate  this  point,  an  input  image  containing  two 
separate  luminance  intensities  is  input  into  a  PCNN  (see  Figure  28).  Without  linking,  the  outputs 
from  level  one  alternate  between  the  two  intensity  levels,  as  can  be  seen  in  Figure  29.  However, 
when  linking  is  added  to  level  1,  the  two  separate  image  intensities  are  close  enough  such  that  they 
“bind”  together  and  pulse  simultaneously.  The  result,  then,  is  an  output  identical  to  the  input  in 
Figure  28. 
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Figure  28.  PCNN  Input  Pattern  with  multiple  luminance  intensity 
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Figure  29.  PCNN  Output  Patterns  at  level  1:  luminance  intensity  =  a)l  b).4  (no  linking) 

For  all  previous  simulations,  the  weighting  for  all  feeding  and  linking  inputs  has  followed  a 
positive  gaussian  profile.  If  a  lateral  inhibition  weighting  profile  such  as  the  following: 


-.125  -.125  -.125 
-.125  1  -.125 

-.125  -.125  -.125 

is  applied  to  level  2  neurons,  the  resulting  interactions  produce  a  series  of  “edge  detectors”.  If 
the  solid  input  from  Figure  24  is  fed  into  such  a  configuration,  the  output  is  a  series  of  pulses 
corresponding  to  the  pattern’s  outline,  as  shown  in  Figure  30. 


3.5.2  Simulation  of  Apparent  Motion.  As  was  discussed  earlier,  Grossberg’s  “Global 
Motion  Layer”  operates  under  a  “winner  take  all”  scenario  which  utilizes  a  comparison  of  individual 
cell  activity  levels  at  discrete  time  increments.  However,  as  was  demonstrated  in  Figure  25,  internal 
activity  for  a  given  cell  is  transferred  to  other  cells  through  a  series  of  output  pulses.  As  a  result, 
any  demonstration  of  apparent  motion  using  the  PCNN  model  must  utilize  pulse  rate  information 
in  its  implementation. 
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Figure  30.  PCNN  Level  2  Output  (feeding  radius  =  1,  linking  radius  =  0,  lateral  inhibition  weight¬ 
ing  profile) 

To  demonstrate  apparent  motion,  two  point  source  inputs  (shown  in  Figure  31)  are  presented 
at  time  iterations  i  =  1  and  i  =  50  with  the  associated  model  parameters  presented  in  Table  2.  In 
this  scenario,  each  input  in  the  second  PCNN  layer  has  a  receptive  field  equal  to  9  such  that  its 
feeding  radius  is  set  equal  to  4.  For  the  simulation  runs  shown  here,  no  linking  inputs  are  used. 
A  sample  of  the  output  pulses  at  iteration  intervals  i=20,  60,  and  75,  is  shown  in  Figure  32.  For 
a  complete  listing  of  the  level  2  apparent  motion  output  pulses,  see  Appendix  D.  As  can  be 
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Figure  31.  Apparent  Motion  Point  Source  Inputs:  a)i=0  b)i=50 


seen  in  the  successive  pulse  outputs  from  level  2  neurons,  output  is  not  limited  to  those  neurons 
which  correspond  to  stimulated  neurons  at  the  input  level.  Instead,  pulse  output  is  present  up 
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Table  2.  Apparent  Motion  Simulation  Parameters 
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Figure  32.  PCNN  Apparent  Motion  Output:  a)i=20  b)i=60  c)i=75 


35 


to  a  distance  of  4  neurons.  (This  distance  corresponds  to  the  feeding  radius  for  level  2  neurons.) 
The  difference  in  the  output  is,  as  mentioned  earlier,  exhibited  in  the  pulse  rate  of  each  neuron. 
Temporally,  the  center  of  pulsing  activity  at  the  output  level  (level  2)  moves  from  the  location  of 
the  first  input  to  the  location  of  the  second  input.  Since  pulse  activity  is  centered  briefly  over  lower 
level  neurons  whch  are  never  stimulated,  the  simulation  is  interpreted  to  exhibit  apparent  motion. 

3.6  Conclusion 

In  Chapter  2,  the  presence  of  a  temporal  delay  was  shown  to  play  a  major  role  in  perceptual 
processing  in  the  brain.  Specifically,  the  delay  was  shown  to  allow  both  pre-attentive  and  post- 
attentive  stimuli  to  have  an  influence  upon  perceived  stimuli.  The  physiological  basis  for  how 
this  temporal  delay  might  occur  in  a  neurological  framework  was  largely  ignored.  However,  the 
Pulse  Coupled  Neural  Network  developed  by  Eckhorn  et.  al.,  and  implemented  here  in  Chapter 
3,  allows  the  physiological  foundation  for  temporal  delay  to  be  re-addressed.  Specifically,  the 
PCNN  has  been  shown  to  translate  stimulus  inputs  into  pulse-based  temporal  representations. 
As  a  result,  variations  in  input  frequency  and  amplitude  translate  to  unique  pulsed-code  output. 
More  important,  however,  is  the  ability  of  the  PCNN  to  simulate  the  build-up  of  internal  activity  in 
individual  neurons  through  the  feeding  and  linking  inputs  of  other  neurons.  It  is  this  build-up,  prior 
to  pulsed  output,  which  allows  the  PCNN  to  produce  pulsed  output  from  neurons  which  received 
no  direct  stimulus  input  (ie.,  apparent  motion).  Through  modification  and  implementation  of  the 
model  neuron,  the  PCNN  has  been  shown  to  provide  a  physiological  motivation  for  the  temporal 
influence  upon  stimuli  perception.  In  Chapter  4  which  follows,  the  degree  to  which  this  temporal 
influence  is  present  in  both  uni-modal  and  multi-modal  sensory  processing  will  be  examined. 
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IV.  MultiModal  Noncausality 


4-1  Background:  Neurophysiological  Timing  in  the  Brain 

Michael  S.  Gazzaniga,  commenting  upon  the  nature  of  the  conscious  experience  in  the  ex¬ 
istence  of  unique  human  mental  capacities,  once  remarked, “..our  awareness,  our  consciousness  of 
these  capacities,  is  nothing  more  or  less  than  a  feeling  about  them”  (8)  If  such  a  premise  is  ac¬ 
cepted,  current  neurological  and  psychological  research  can  offer  little  more  than  conjecture  as  to 
the  source  and  implementation  of  these  “feelings”  within  the  brain.  However,  the  study  of  con¬ 
scious/unconscious  experience  need  not  be  shelved  just  yet.  The  “quality”  (16)  of  consciousness 
may  remain  beyond  the  reach  of  current  scientific  capabilities,  but  its  presence  (or  conversely,  ab¬ 
sence)  has  been  documented  through  both  physiological  and  psychological  experimentation.  Such 
research,  principally  the  work  of  Dr  Benjamin  Libet  and  his  associates,  has  provided  a  physiologi¬ 
cal  foundation  for  the  existence  of  a  neuro-time  factor  in  association  with  the  presence/absence  of 
conscious  experience. 

In  1963,  Dr  Libet  presented  results  detailing  the  presence  of  a  minimum  threshold  for  con¬ 
scious  experience(21)(19)(18)(23)(22).  The  conscious  threshold  was  determined  by  stimulating  the 
somatosensory  cortex  with  pulse  trains  at  the  liminal  (threshold)  intensity  necessary  for  any  sen¬ 
sation  response(21).  Evoked  responses  were  measured  for  a  wide  variety  of  pulse  stimuli.  However, 
subjects  experienced  conscious  sensations  only  when  the  applied  pulse  train  durations  equaled  or 
exceeded  approximately  500ms. (23)  (Note:  Libet  stated  that  the  500ms  temporal  requirement  was 
not  rigid;  stronger  stimuli  with  shorter  durations  were  also  shown  to  be  effective(22)).  Single,  long- 
duration  pulses  with  increased  amplitudes  were  unable  to  produce  the  conscious  sensation(19).  The 
significance  of  this  result  was  threefold:  l)the  presence  of  evoked  potentials  without  the  associated 
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conscious  sensation  provided  a  possible  explanation  for  the  existence  of  “subliminal  perception” 
as  well  as  the  phenomenon  of  “blindsight”  as  described  by  Weiskrantz(18)(30);  2)the  requirement 
for  a  500ms  time  duration  predicated  the  existence  of  a  neuronal  delay  above  and  beyond  the  de¬ 
lays  previously  thought  to  exist  due  to  pulse  transmission  and  accumulation(18);  3)the  need  for  a 
minimum  duration  of  pulse  activity  added  weight  to  the  theory  that  the  combination  of  the  two 
factors  (time,  pulse  activity)  provided  the  coding  for  neural  conscious  activity(22).  The  Pulse  Cou¬ 
pled  Neural  Network  proposed  by  Eicken  et  al.  and  described  in  detail  in  Chapter  2,  provides  a 
promising  realization  for  such  a  coding  scheme.  Based  upon  data  collected  from  the  striate  cortex 
of  a  cat,  the  network  encodes  information  through  neuron  pulse  rates(6).  (As  an  added  benefit, 
the  internal  cell  activity  level  for  individual  neurons  can  be  shown  to  grow  or  decay  over  time  as 
a  product  of  feeding  and/or  linking  inputs.  As  a  result,  conscious  sensation  can  be  shown  to  exist 
(ie.,  fire)  only  after  unconscious/conscious  feeding  and  linking  inputs  have  produced  the  necessary 
internal  cell  activity  level.) 

Dr.  Libet’s  discovery  of  a  minimum  conscious  threshold  and  an  associated  pulse  train  duration 
led  to  an  increased  study  of  the  impact  of  the  neural  time  factor  upon  time  perception.  Specifically, 
he  found  that  there  was  “..no  appreciable  delay  for  the  subjective  timing  of  a  normally  arriving 
sensory  input”  (20).  Subsequent  experiments  utilizing  relative  ordering  of  subjective  timing  enabled 
Libet  to  conclude  that  the  delay  preceeding  consciousness  of  activity  was  followed  by  an  automatic 
subjective  referral  of  the  experience  backwards  in  time  to  the  approximate  point  when  the  stimulus 
originally  occurred  -  the  subjective  present. (20)(4).  According  to  Libet,  a  fast  sensory  message 
from  the  original  cortical  response  provided  the  temporal  anchor  point(20).  The  result,  then,  was 
a  response  which  subjectively  occurred  without  delay.  The  diagram  used  by  Libet  to  explain  this 
phenomenon  is  reprinted  in  Figure  33(22). 
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Figure  33.  Subjective  Backwards  Referral  in  Time 
4-2  Implications  of  Subjective  Timing  in  Conscious  Perception 

4-2.1  Uni-modal  Stimuli(uni-modal  temporal  succession).  In  the  course  of  his  research, 
Dr.  Libet  hinted  at  the  possibility  that  unconscious  experiences,  constrained  by  a  shorter  latency 
for  development,  might  have  the  ability  to  participate  in  and  thereby  affect  cerebral  activities  prior 
to  conscious  awareness  of  these  activities(18).  Under  such  a  scenario,  two  temporally  separate 
conscious  somatosensory  stimuli  could  provide  simultaneous  inputs  to  higher  level  cerebral  pro¬ 
cessing.  Evidence  for  just  such  a  situation  was  demonstrated  back  in  1970  by  Richard  and  Roslyn 
Warren(29).  As  part  of  a  larger  research  effort  into  the  dynamics  of  audio  perception,  Warren  and 
Warren  found  that  the  perception  of  words  presented  earlier  in  a  sentence  could  be  determined 
by  the  context  of  the  latter  half  of  the  sentence(29).  Prom  this  outcome,  both  scientists  surmised 
that  the  ability  to  achieve  correct  phonemic  restoration  was  the  result  of  “...storing  the  incomplete 
information  until  the  necessary  context  [was]  supplied  so  that  the  required  phoneme  [could]  be 
synthesized(29).”  Acceptance  of  a  “subjective  backwards  referral”  operation  produces  a  slightly 
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different  interpretation  of  the  Warrens’  experimental  results.  If  conscious  awareness  is  delayed  some 
time  after  occurrence  of  the  audio  stimuli,  then  the  conscious  perception  of  the  stimuli  may  be  in¬ 
fluenced  by  other  stimuli  which  occur  during  the  delay.  If  the  conscious  awareness  of  the  stimuli 
is  then  subjectively  moved  backwards  in  time,  then  the  result  would  be  conscious  perception  of  a 
stimulus  which  differs  from  the  original  sensory  stimulus  input. 

4.2.2  Multi-Modal  Stimuli(multi-modal  temporal  succession).  During  the  course  of  his 
research.  Dr  Libet  noted  that  the  subjective  “backwards  referral  in  time”  capability  provides 
a  corrective  mechanism  for  the  temporal  differences  observed  in  the  processing  of  simultaneous 
multi-sensory  inputs(18).  Without  such  a  capability,  stimuli-dependent  transmission  and  process¬ 
ing  delays  would  lead  to  a  temporal  sequence  of  conscious  sensations  (figure  34).  By  subjectively 
referring  the  conscious  perception  of  the  sensations  back  to  their  originating  times(ie.,  the  original 
cortical  responses),  a  sense  of  simultaneous  stimulation  is  achieved(Figure  35).  What,  then,  is 


Perceptual  Time 

Figure  34.  With-Out  Subjective  Backwards  Referral  in  Time 

the  nature  of  stimuli  perception  if  two  stimuli  are  applied  with  some  temporal  delay?  Research 
into  this  area  has  demonstrated  the  inability  of  test  subjects  to  discriminate  the  order  of  unimodal 
stimuli  applied  in  succession  with  very  small  temporal  delays  (on  the  order  of  tens  of  milliseconds) 
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Figure  35.  With  Subjective  Backwards  Referral  in  Time 

(29).  In  affect,  the  stimuli  are  perceived  as  having  occurred  in  a  simultaneous  fashion.  Such  a 
response  would  seem  to  support  the  existence  of  a  temporal  perceptual  window  -  multiple  stimuli 
occuring  anywhere  within  this  window  are  processed  in  a  perceptual  sense  as  simultaneous.  The 
maximum  allowable  difference  in  time  between  two  (or  more)  stimuli,  which  would  still  result  in 
simultaneous  perception,  would  then  define  the  temporal  tolerance  of  this  perceptual  window.  A 
possible  graphical  representation  of  such  a  window  is  represented  in  Figure  36.  The  implications  for 


Perceptual  Time 


Figure  36.  With  Subjective  Backwards  Referral  in  Time 


the  existence  of  a  perceptual  window  are  tremendous.  Information  processing  which  utilized  such 
information  would,  in  effect,  operate  in  a  non-causal  manner  (with  respect  to  current  perception). 
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One  such  processing  function,  multi-modal  sensory  integration,  has  been  shown  to  exist  in  the 
thalamus(nucleus  dorsolateralis  posterior  thalami)  of  the  pigeon  (17), as  well  as  the  cortex  of  the 
cat  anterior  ectosylvian  sulcus  (28).  Neurons  within  these  areas  respond  to  unique  modal  combina¬ 
tions.  The  presence  of  a  sliding  temporal  window  would  alleviate  the  need  for  exact  syncronization 
of  multisensory  inputs  into  these  neurons  and  at  the  same  time,  greatly  enhance  the  robustness 
of  the  overall  system.  To  test  such  a  hypothesis,  a  series  of  multi-modal  experiments  have  been 
devised: 

4-3  MultiModal  Experiments 

4-3.1  Materials  and  Methods.  In  the  procedures  to  be  described  below,  six  AFIT  en¬ 
gineering  students  were  used  as  subjects  in  Experiments  I,  II,  and  II.  All  six  students  undertook 
each  experiment  individually,  one  sitting  per  experiment.  Visual  and  auditory  stimuli  were  pro¬ 
vided  to  each  subject  via  a  monitor  and  headphones  connected  to  a  Zenith  486  computer  using 
a  33Mhz  processor  with  sound  card  installed.  The  subjects  were  instructed  to  sit  perpendicular 
to  the  screen,  approximately  8  inches  from  it.  The  visual  display  seen  by  each  student  was  com¬ 
posed  of  a  640x480  pixel  framed  background,  a  20x20  pixel  black  cross-hair  in  the  center  of  the 
frame,  and  three  graphical  buttons  in  the  lower  right-hand  side  of  the  frame  labeled  “go” ,  “left” , 
and  “right”,  respectively(Figure  37).  Wearing  headphones,  subjects  were  instructed  to  fixate  on 
the  black  cross-hair  and  select  “go”  with  the  attached  “mouse.”  After  a  two  second  delay,  two 
black  5x5  pixel  boxes  appeared  on  each  side  of  the  screen,  separated  by  a  spatial  distance  of  500 
pixels(8.25  inches)  and  a  temporal  delay  (in  milliseconds)  preset  prior  to  commencement  of  the 
test.  The  spatial  separation  of  visual  stimuli  was  used  as  a  means  to  force  each  test  subject  to 
use  his/her  peripheral  vision  when  viewing  the  two  black  boxes.  During  an  investigation  of  the 


42 


spatial  distribution  of  visual  attention,  Mangun  and  Hillyard  observed  the  existence  of  a  gradient 
of  attention  which  corresponded  to  the  measured  distance  from  focused  attention  to  the  location 
of  the  stimulus(24).  This  test  observation  was  used  as  a  means  to  sufficiently  reduce  attentional 
awareness  of  the  two  visual  stimuli  such  that  the  test  subject  would,  if  the  temporal  delay  was 
short  enough,  experience  difficulty  in  selecting  the  stimuli  which  appeared  first.  In  conjunction 
with  the  visual  display,  test  subjects  would  hear  one  of  three  possible  audio  responses;  l)silence, 
2)audio  tones  presented  in  each  ear  corresponding  to  the  order  of  appearance  of  each  visual  block, 
or  3)  audio  tones  in  each  ear  presented  in  reverse  order  from  the  visual  stimuli  order  of  the  black 
boxes.  The  degree  of  modal  temporal  synchronization  ,  (ie.,  the  amount  of  overlap  of  visual  and 
audio  stimuli  in  time)  was  determined  by  a  second  delay  parameter  set  prior  to  initialization  of 
each  test.  Upon  conclusion  of  the  visual/audio  stimuli  presentation,  each  subject  was  instructed  to 
make  a  forced  choice  (left  or  right)  as  to  which  block  appeared  on  the  screen  first.  This  scenario 
was  repeated  (in  each  experiment)  for  each  student  60  times. 


Figure  37.  Visual  Interface  Seen  by  Test  Subjects 

Synchronization  of  visual  and  auditory  stimuli  was  achieved  using  a  software  program  writ¬ 
ten  in  the  C  program  language  implementing  a  series  of  application  independent  (API)  Windows 
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routines.  Utilization  of  the  windows  environment  greatly  simplified  the  graphical  interface  required 
for  the  visual  portion  of  the  experiment.  In  addition,  such  an  environment  allowed  for  real-time 
variable  modification  within  the  program  itself. 

Upon  initialization  of  the  Windows  program,  a  calibration  subroutine  was  called  which  es¬ 
tablished  the  number  of  loops  executed  per  millisecond  by  the  processor.  This  parameter  was 
subsequently  used  throughout  the  program  to  set  the  number  of  “do  nothing”  loops  necessary  for  a 
prescribed  delay  of  x  milliseconds.  For  example,  a  call  to  the  CalibrateTimer  routine  might  return 
a  loops-per-millisecond  value  of  12,303.  A  delay  of  30ms  would  then  correspond  to: 

number  of  delay  loops  =  12, 303  Loops  —  per  —  millisecond  x  30ms  =  369090  (9) 

This  calibrated  value  was  then  multiplied  by  two  values:  the  desired  millisecond-delay  between  the 
appearance  of  each  black  box,  and  the  desired  millisecond  delay  between  visual  and  audio  stimuli. 

When  the  “go”  button  was  selected,  one  of  three  possible  integer  values  (0,  1,  or  2)  was 
selected  using  a  built-in  pseudo-random  number  generator.  The  random  number  generator  itself  was 
initialized  or  “seeded”  using  a  generic  Windows  “randomize”  routine  called  during  the  callibration 
sequence  described  above.  If  a  “0”  was  generated,  an  experimental  trial  without  audio  tone  was 
selected.  If  a  “1”  was  generated,  an  experimental  trial  in  which  both  visual  and  audio  stimuli 
occured  in  the  same  order  was  selected.  Finally,  if  a  “2”  was  randomly  generated,  an  experimental 
trial  in  which  visual  and  audio  stimuli  occurred  in  temporally-reversed  order  was  selected.  No 
matter  which  case  was  randomly  selected,  the  initial  occurrence  of  visual  and  audio  stimuli  (ie., 
left  or  right)  was  stored  -  along  with  the  selection  made  by  the  test  subject.  Upon  conclusion  of 
each  test  (60  trials),  a  summary  of  all  pertinent  data  was  made  available  for  analysis. 
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^.3.2  Verification  of  Test  Set-Up:  Due  to  the  need  for  accurate  temporal  measurements 
within  the  multi-modal  experiment,  as  well  as  the  nature  of  the  hardware  used  in  its  implementation, 
a  test  set-up  verification  procedure  was  conceived  and  carried  out  prior  to  actual  experimentation. 
Specifically,  the  actual  delay  (in  milliseconds)  between  onset  of  visual  stimuli  and  auditory  stimuli 
was  measured  and  compared  at  regular  intervals  to  the  delay  calculated  within  the  software.  The 
mean  and  standard  deviation  of  the  delay  measured  at  each  interval  is  provided  in  Table  3.  (See 
appendix  F  for  a  complete  description  of  the  test  setup  verification  procedure  and  the  data  obtained 
from  this  procedure).  As  can  be  seen  in  the  table,  the  variation  in  recorded  measurements  across 


Table  3.  Mean  and  Standard  Deviation  of  Measured  Delay  Intervals 


Calculated  Delay 

Measurement 

50 

40 

30 

20 

10 

0 

-10 

-20 

mean 

39.2 

44.7 

30.0 

31.7 

13.3 

-3 

-17.2 

-43.2 

standard  deviation 

7.2 

7.8 

8.6 

8.4 

6.1 

8.1 

17.5 

visual-auditory  delay  intervals  is  significant.  Conceivably,  this  wide  dispersal  could  render  several  of 
the  planned  test  intervals  redundant  if  the  differences  between  perspective  neighboring  mean  values 
are  found  to  be  statistically  insignificant.  To  test  this  possibility,  a  pair-wise  Test  of  Significance 
was  performed  among  all  delay  intervals  using  a  level  of  significance  of  5%.  The  results  obtained 
from  this  test,  presented  in  Table  29  of  Appendix  F,  found  all  neighboring  interval  means  to  be 
statistically  significant  -  except  the  mean  values  for  test  intervals  at  20  and  30  ms.  As  a  result, 
test  data  collected  at  both  intervals  must  be  considered  as  one  set  of  output. 

As  can  be  seen  in  Figure  38,  the  difference  between  the  expected  and  measured  delay  increases 
as  a  function  of  the  length  of  the  delay  itself.  Such  a  trend  can  be  attributed  to  the  use  of 
the  Windows  graphical  interface.  Windows  is  an  event-driven  environment;  as  such,  the  system 
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Figure  38.  Test  Setup  Timing:  Measured  vs  Expected  Modal  Delays  (in  ms) 


periodically  “borrows”  control  of  the  processor  to  poll  for  system  messages.  Since  this  polling  is 
ongoing,  increased  use  of  the  processor  (in  terms  of  time)  leads  to  an  increased  “chunk”  of  processor 
time  devoted  to  message  checking. 


^.J^  Experiment  I 

4-4-i  Purpose:  The  purpose  of  Experiment  I  was  to  verify  equal  selection  probability  for 
“left”  and  “right”  choices  when  presented  with  visual  stimuli  which  occur  simultaneously. 

4.4-^  Method:  The  delay  between  left  and  right  auditory  stimuli  was  set  equal  to  zero  so 
that  the  test  subject  would  hear  only  one  auditory  tone  in  each  ear  simultaneously.  In  addition,  the 
delay  between  the  onset  of  the  1st  and  2nd  visual  stimuli  (black  boxes)  was  set  equal  to  zero.  Finally, 
the  delay  between  visual  and  auditory  stimuli  was  set  equal  to  zero,  which  produced  a  delay  of 
plus/minus  10ms  between  the  onset  of  the  multi-modal  stimuli.  A  spatial-temporal  representation 
of  this  setup  is  shown  in  Figure  39. 
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Figure  39.  Experiment  I:  visual  and  auditory  stimuli  presented  simultaneously;  visual-auditory 
delay=0ms,  visual-visual  delay=0ms 

4-4-3  Results:  Psycho-sensory  test  data  obtained  from  six  test  subjects  are  presented  in 

Table  4.  A  pair-wise  mean  significance  test  using  a  confidence  factor  of  5%  was  applied  to  the 
test  data  to  determine  whether  or  not  the  observed  mean  values  were  statistically  the  same.  The 
results  from  this  test  are  presented  in  Table  5.  Rejection  of  the  “Null  hypothesis”  is  interpreted  to 
denote  the  existence  of  a  statistical  difference  between  the  data  pair  being  compared.  (Note:  The 
results  shown  have  been  adjusted  to  take  into  account  the  “Alpha  Creep”  observed  to  occur  when 
comparisons  among  multiple  data  means  are  made(14)).  As  can  be  seen  in  the  data,  test  subjects 

Table  4.  Experiment  I:  visual-visual  delay  =  0ms,  visual-auditory  delay  =  0ms 


Percentage  Correct(out  of  30) 

Test  Subject 

guessed  right 

guessed  left 

1 

50 

50 

2 

53 

46 

3 

43 

56 

4 

36 

63 

5 

33 

66 

6 

46 

53 

mean 

43.9 

56.1 

standard  deviation 

7.7 

7.7 

chose  “left”  slightly  more  often  than  “right.”  However,  due  to  the  wide  variation  in  test  data,  the 
difference  in  means  is  deemed  statistically  insignificant. 
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Table  5.  Pair-wise  Test  of  Significance,  visual- visual  delay  =  Oms,  visual-auditory  delay  =  0ms 


Configuration  1 

Configuration  2 

T 

Reject  NULL  Hypothesis? 

guessed  right 

guessed  left 

-1.9285 

NO 

4-5  Experiment  II 

4-5.1  Purpose:  The  purpose  of  Experiment  II  was  to  investigate  and  bound  the  temporal 
perception  of  visual  stimuli  both  with  and  without  the  addition  of  auditory  stimuli. 

4-5.2  Method:  The  delay  between  visual  and  auditory  stimuli  was  set  equal  to  zero, 

which  produced  a  delay  of  plus/minus  10ms  between  the  onset  of  the  multi-modal  stimuli.  The 
delay  between  the  onset  of  the  1st  and  2nd  visual  stimuli  (black  boxes)  was  then  increased,  in  10ms 
increments,  from  Oms  (ie.,  simultaneous)  upward  to  40ms.  Spatial-temporal  representations  of  each 
possible  stimulus  presentation  is  shown  in  Figures  40  through  42. 


Figure  40.  Experiment  II:  visual  and  auditory  stimuli  presented  in  order;  visual-auditory  de- 
lay=0ms,  visual-visual  delay=variable 
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Figure  41. 


Figure  42. 


Ps 


Experiment  II:  visual  and  auditory  stimuli  presented  out  of  order;  visual-auditory 
delay=0ms,  visual-visual  delay=variable 


L 

AUDITORY 


Experiment  II:  visual  stimuli  presented  without  accessory  auditory  stimuli;  visual- 
visual  delay=variable 
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4-5.3  Results:  Psycho-sensory  test  data  obtained  from  six  test  subjects  at  visual  stimulus 
delays  of  0,  10,  20,  and  30  milliseconds,  respectively,  are  presented  (along  with  corresponding  means 
and  standard  deviations)  in  Tables  6  through  9. 


Table  6.  Experiment  II:  visual  delay=0ms 


Percentage  Correct(out  of  20) 

Test  Subject 

silent 

in-order 

out-of-order 

1 

10 

2 

25 

3 

90 

15 

4 

85 

5 

5 

80 

30 

6 

50 

35 

mean 

49.2 

standard  deviation 

5.9 

14.3 

11.8 

Table  7.  Experiment  II;  visual  delay=20ms 


Percentage  Correctfout  of  20) 

Test  Subject 

silent 

out-of-order 

1 

iim 

85 

30 

2 

20 

3 

75 

95 

20 

4 

45 

65 

30 

5 

75 

95 

30 

6 

55 

85 

25 

mean 

69.2 

87.6 

25.9 

standard  deviation 

17.4 

12.6 

4.9 

A  pair-wise  mean  significance  test  using  a  confidence  factor  of  5%  was  applied  to  each  set 
of  data  to  determine  the  validity  of  separation  observed  in  the  data  mean  values.  The  results, 
presented  in  Tables  10  through  13,  confirm  the  existence  of  cross-modal  effects  upon  perception. 
To  illustrate  the  extent  of  these  effects  over  time,  the  median  correct  percentages  for  all  three 
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Table  8.  Experiment  II:  visual  delay=30ms 


Percentage  Correct  (out  of  20) 

Test  Subject 

silent 

in-order 

out-of-order 

1 

m 

95 

55 

2 

100 

50 

3 

ifQii 

90 

25 

4 

50 

75 

40 

4 

75 

40 

5 

95 

90 

50 

6 

100 

95 

85 

mean 

77.5 

90.8 

50.8 

standard  deviation 

23.0 

8.6 

19.9 

Table  9.  Experiment  II:  visual  delay =40ms 


Percentage  Correct(out  of  20) 

Test  Subject 

silent 

in-order 

out-of-order 

1 

85 

100 

85 

2 

80 

90 

65 

3 

80 

95 

85 

4 

95 

100 

50 

5 

90 

100 

60 

6 

100 

95 

65 

mean 

88.3 

96.7 

68.3 

standard  deviation 

8.2 

4.1 

14.0 

Table  10.  Pair-wise  Test  of  Significance,  visual  delay  =  0ms 


Configuration  1 

Configuration  2 

T 

Reject  NULL  Hypothesis? 

silent 

in  order 

-3.9528 

YES 

silent 

out  of  order 

5.5760 

YES 

in  order 

out  of  order 

5.6362 

YES 
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Table  11.  Pair-wise  Test  of  Significance,  visual  delay  =  20ms 


Configuration  1 

Configuration  2 

T 

Reject  NULL  Hypothesis  ? 

silent 

in  order 

-5.5 

YES 

silent 

out  of  order 

5.1387 

YES 

in  order 

out  of  order 

9.4285 

YES 

Table  12.  Pair-wise  Test  of  Significance,  visual  delay  =  30ms 


Configuration  1 

Configuration  2 

T 

Reject  NULL  Hypothesis? 

silent 

in  order 

-1.807 

NO 

silent 

out  of  order 

4.3386 

YES 

in  order 

out  of  order 

5.3936 

YES 

Table  13.  Pair-wise  Test  of  Significance,  visual  delay  =  40ms 


Configuration  1 

Configuration  2 

T 

Reject  NULL  Hypothesis? 

silent 

in  order 

-5.8797 

YES 

silent 

out  of  order 

1.8279 

NO 

in  order 

out  of  order 

3.7251 

YES 
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test  scenarios  were  plotted  at  the  0,  10,  20,  and  30  millisecond  delay  intervals,  respectively.  The 
resulting  graph  is  presented  in  Figure  43.  As  shown  in  the  graph,  the  impact  of  auditory  stimuli 


Visual  InterStimulus  Interval(ISI)  Effect  Upon  Choice 


Figure  43.  Experiment  II:  Effects  of  Inter  Stimulus  Interval(ISI)  for  visual  stimuli  applied  in  the 
presence  of  auditory  stimuli 

upon  visual  perception  is  diminished  as  the  delay  between  the  onset  of  visual  stimuli  is  increased. 
The  same  effect  is  experienced  when  viewing  a  movie  or  television  program  whose  audio  signal  is 
out-of-sync  with  the  accompanying  video  signal;  the  descrepancy  is  often  not  apparent  until  the 
temporal  gap  between  audio  and  visual  presentation  is  extensive.  This  result  appears  to  confirm 
previous  studies  on  Intersensory  Discrepancy:  as  the  information  from  one  sense  becomes  more 
discrepant  from  the  information  from  the  second  sense,  the  influence  of  the  former  on  the  latter 
decreases  (1).  When  viewed  from  the  same  perspective  as  Dr.  Libet’s  experiments,  the  diminished 
effect  can  be  attributed  to  temporal  activity  which  “spills  over”  a  given  perceptual  window  of 
simultaneous  activity. 

The  effects  of  accessory  auditory  stimuli  have  all  been  causal  throughout  Experiment  II; 
the  onset  of  auditory  stimuli  is  never  delayed  to  occur  after  the  onset  of  the  first  visual  stimuli. 
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To  determine  the  effects  of  delayed  auditory  stimuli  onset  in  relation  to  visual  stimuli  onset,  the 
following  experiment  has  been  devised. 

4-6  Experiment  III 

4.6.1  Purpose:  The  purpose  of  Experiment  III  was  to  investigate  and  bound  the  effects 

of  delayed  auditory  stimuli  upon  perception  of  visual  stimuli. 

4.6.2  Method:  The  delay  between  the  onset  of  the  1st  and  2nd  visual  stimuli  (black 

boxes)  was  set  equal  to  20ms.  (This  value  was  shown  in  Experiment  II  to  produce  the  greatest 
variation  in  regards  to  correct  choices  made  without  auditory  inputs  and  auditory  inputs  presented 
in  reverse  order  from  the  visual  stimuli).  The  delay  between  onset  of  the  first  visual  stimulus  and 
auditory  tone  was  then  increased,  in  10ms  increments,  from  0ms  (ie.,  simultaneous  onset)  upwards 
to  60ms.  Spatial-temporal  representation  of  each  possible  stimulus  presentation  is  shown  in  Figures 
44  through  46. 


Figure  44.  Experiment  III:  visual  and  auditory  stimuli  presented  in  order;  visual-auditory  de- 
lay=variable,  visual-visual  delay=20ms 
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Figure  45.  Experiment  III:  visual  and  auditory  stimuli  presented  out  of  order;  visual-auditory 
delay=variable,  visual-visual  delay=20ms 


VISUAL 


L 


R 


I 

AUDITORY 


R 


Figure  46. 


Experiment  III:  visual  stimuli  presented  without  accessory  auditory  stimuli;  visual- 
visual  delay=20ms 
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4-6.3  Results:  Psycho-sensory  test  data  obtained  from  six  test  subjects  with  visual- 

auditory  stimulus  onset  delays  of  0,  10,  20,  30,  40,  50,  and  60  milliseconds,  respectively,  are  pre¬ 
sented  (along  with  corresponding  means  and  standard  deviations)  in  Tables  14  through  20.  A 


Table  14.  Experiment  III:  visual  audio  onset  delay=0ms 


Percentage  Correct(out  of  20) 

Test  Subject 

silent 

in-order 

out-of-order 

1 

70 

85 

30 

2 

95 

100 

20 

3 

75 

95 

20 

4 

45 

65 

30 

5 

75 

95 

30 

6 

55 

85 

25 

mean 

69.2 

87.6 

25.9 

standard  deviation 

17.4 

12.6 

4.9 

Table  15.  Experiment  III:  visual-audio  onset  delay=10ms 


Percentage  Correct(out  of  20) 

Test  Subject 

silent 

in-order 

out-of-order 

1 

90 

100 

30 

2 

75 

100 

10 

3 

80 

55 

35 

4 

55 

85 

60 

5 

70 

80 

40 

6 

60 

75 

55 

mean 

71.4 

80.0 

39.3 

standard  deviation 

11.8 

16.8 

16.7 

pair-wise  mean  significance  test  using  a  confidence  factor  of  5%  was  applied  to  each  set  of  data  to 
determine  the  validity  of  separation  observed  in  the  date  mean  values.  The  results  for  each  delay 
interval  can  be  seen  in  Tables  21  through  27. 
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Table  16.  Experiment  III:  visual-audio  onset  delay=20ms 


Percentage  Correct  (out  of  20) 

Test  Subject 

silent 

in-order 

out-of-order 

1 

75 

2 

65 

90 

55 

3 

75 

90 

4 

75 

75 

65 

5 

50 

55 

35 

6 

80 

100 

60 

7 

80 

95 

40 

mean 

69.3 

82.6 

49.3 

standard  deviation 

11.3 

15.5 

14.0 

Table  17.  Experiment  III:  visual-audio  onset  delay=30ms 


Percentage  Correctfout  of  20) 

Test  Subject 

in-order 

out-of-order 

1 

D 

20 

2 

85 

100 

30 

3 

65 

90 

35 

4 

55 

50 

50 

5 

55 

70 

55 

6 

65 

75 

80 

mean 

67.5 

45 

standard  deviation 

10.8 

21.4 
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Table  18.  Experiment  III;  visual-audio  onset  delay=40ms 


Percentage  Correct(out  of  20) 
Test  Subject  silent  in-order  out-of-order 


Table  19.  Experiment  III:  visual-audio  onset  delay=50ms 


Test  Subject 


Percentage  Correctf out  of  20) 


silent  in-order  out-of-order 


Table  20.  Experiment  III:  visual-audio  onset  delay=60ms 


Percentage  Correctfout  of  20) 

Test  Subject 

silent 

in-order 

out-of-order 

1 

80 

2 

WBM 

90 

75 

3 

95 

35 

4 

mil 

65 

5 

40 

65 

6 

40 

mean 

67.5 

70.0 

55.8 

standard  deviation 

12.1 

22.1 

16.6 
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Table  21.  Pairwise  Test  of  Significance,  visual-auditory  delay  =  0ms,  visual  delay  =  20ms 


Configuration  1 

Configuration  2 

T 

Reject  NULL  Hypothesis? 

silent 

in  order 

-5.5 

YES 

silent 

out  of  order 

5.1387 

YES 

in  order 

out  of  order 

9.4285 

YES 

Table  22.  Pairwise  Test  of  Significance,  visual-auditory  delay  =  10ms,  visual  delay  =  20ms 


Configuration  1 

Configuration  2 

T 

Reject  NULL  Hypothesis? 

silent 

in  order 

-1.2163 

NO 

silent 

out  of  order 

3.217 

NO 

in  order 

out  of  order 

3.8 

YES 

Table  23.  Pairwise  Test  of  Significance,  visual-auditory  delay  =  20ms,  visual  delay  =  20ms 


Configuration  1 

Configuration  2 

T 

Reject  NULL  Hypothesis? 

silent 

in  order 

-4.2136 

YES 

silent 

out  of  order 

3.1909 

NO 

in  order 

out  of  order 

4.5508 

YES 

Table  24.  Pairwise  Test  of  Significance,  visual-auditory  delay  =  30ms,  visual  delay  =  20ms 


Configuration  1 

Configuration  2 

T 

Reject  NULL  Hypothesis? 

silent 

in  order 

-3.1 

NO 

silent 

out  of  order 

1.9 

NO 

in  order 

out  of  order 

2.48 

NO 

Table  25.  Pairwise  Test  of  Significance,  visual-auditory  delay  =  40ms,  visual  delay  =  20ms 


Configuration  1 

Configuration  2 

T 

Reject  NULL  Hypothesis? 

silent 

in  order 

-2.4 

NO 

silent 

out  of  order 

3.1 

NO 

in  order 

out  of  order 

4.0 

YES 
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Table  26.  Pairwise  Test  of  Significance,  visual-auditory  delay  =  50ms,  visual  delay  =  20ms 


Configuration  1 

Configuration  2 

T 

Reject  NULL  Hypothesis? 

silent 

in  order 

-0.1 

NO 

silent 

out  of  order 

2.1 

NO 

in  order 

out  of  order 

1.9 

NO 

Table  27.  Pairwise  Test  of  Significance,  visual-auditory  delay  =  60ms,  visual  delay  =  20ms 


Configuration  1 

Configuration  2 

T 

Reject  NULL  Hypothesis? 

silent 

in  order 

-0.4 

NO 

silent 

out  of  order 

1.5 

NO 

in  order 

out  of  order 

1.3 

NO 

As  shown  in  the  data,  the  means  associated  with  each  test  presentation  can  be  considered 
statistically  significant  until  the  delay  between  initial  visual  and  auditory  stimuli  reaches  50  ms. 
At  this  point,  the  large  variation  in  test  data  reduces  the  significance  of  the  observed  difference  in 
means.  With  these  limitations  in  mind,  a  graph  illustrating  the  effects  of  temporal  delay  between 
visual  and  auditory  stimuli  onset  is  presented  in  Figure  47.  The  sharp  drop  observed  at  the  40 
ms  delay  interval  is  due  largely  to  one  test  subject’s  unusually  high  susceptability  to  the  effects  of 
out-of-order  stimuli  presentation,  (refer  back  to  Table  18) 

As  a  result  of  the  variability  of  inter-modal  intevals  observed  during  test  setup  verification, 
the  measurements  made  throughout  Experiment  III  can  not  be  used  to  determine  exact  temporal 
boundaries  for  the  relationship  between  visual  and  auditory  stimuli.  However,  the  data  is  useful 
in  that  it  establishes  a  notable  trend:  accessory  auditory  stimuli  have  maximum  impact  upon 
visual  choice  if  applied  simultaneously  (ie.,  inter-modal  stimulus  onset  delay  =  0).  This  effect  is 
diminished  but  still  evident  as  the  delay  between  visual  and  auditory  stimuli  is  increased.  As 
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Delayed  Auditory  Onset  Effect  Upon  Choice 


Figure  47.  Experiment  III:  Effects  of  Inter-Modal  Stimulus  Interval(IMSI)  (for  visual- auditory 
stimuli  onset)  upon  visual  choice  response 

illustrated  in  Figure  47,  the  decrease  in  auditory  influence  is  gradual.  Specifically,  even  when  the 
auditory  stimulus  is  initiated  40  ms  after  the  first  visual  stimulus,  the  auditory  stimulus  can  alter 
the  perception  of  the  visual  stimulus.  This  observed  result  suggests  that  perceived  future  inputs 
can  impact  current  perception.  Such  data  once  again  reinforces  the  idea  that  the  perceptual  “now” 
is  not  instantaneous,  but  encompasses  a  “window”  in  a  neurological  time  frame.  The  temporal 
dimensions  of  this  window  establish  the  temporal  resolution  of  the  mind. 
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V.  Conclusions 


Throughtout  this  research  effort,  an  emphasis  has  been  placed  upon  the  investigation  of 
the  neuronal-level  propogation  of  visual  stimulus  information  and  the  temporal  effects  associated 
with  this  propagation.  During  the  course  of  this  investigation,  a  non-causal  link  was  shown  to 
exist  between  Grossberg’s  Motion  Oriented  Contrast  Filter  and  AFIT’s  Adaptive  Retinal  Model. 
This  link  was  further  developed  through  the  simulation  of  Apparent  Motion  using  a  two-level 
Pulse  Coupled  Neural  Network.  (Simulation  of  the  visual  illusion  was  achieved  by  modifying  the 
PCNN  to  implement  a  “global  layer”  analogous  to  the  level  5  cells  of  a  MOC  filter).  Finally,  the 
investigation  into  temporal  effects  upon  awareness  was  broadened  to  include  a  study  of  the  influence 
of  multi-modal  stimulus  inputs.  Observations  and  conclusions  drawn  from  this  effort  include  the 
following: 

1 .  The  presence  of  a  neuronal  framework  whereby  lower- layer  neurons  feed  into  higher  layers  with 
progressively  larger  feeding  and/or  linking  footprints  leads  to  the  propagation  of  temporal 
delays  in  the  realization  of  outputs  at  the  higher  levels.  In  addition,  the  presence  of  internal 
cell  activity  (at  each  cell)  governed  by  activity  levels  akin  to  a  leaky  integrator(ie.,  existence 
of  a  finite  build-up  and  decay  time  constant)  provides  each  cell  with  its  own  rudimentary 
short  term  “memory.”  When  this  cell  activity  memory  is  coupled  with  “global”  neuronal 
connections,  the  result  is  a  filtered  representation  of  the  external  world.  From  this  perspective, 
the  apparent  motion  phenomenon  might  be  considered  a  “quirk”  in  the  operation  of  the  filter. 

2.  The  Pulse  Coupled  Neural  Network  can  be  configured  to  demonstrate  a  “pulsed”  represen¬ 
tation  of  the  apparent  motion  phenomenon.  This  presentation  provides  a  unique  perspective 
for  possible  interpretation  of  the  visual  illusion. 
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3.  Perceptual  and  physical  time  are  not  necessarily  the  same.  In  fact,  the  global  interactions 


described  above  are  governed  by  the  time  constants  of  activity  levels  internal  to  each  neuron 
at  every  level.  As  a  result,  the  precise  temporal  information  inherent  in  the  arrival  of  a 
stimulus  input  is  lost  during  the  propagation  of  that  information  through  the  brain.  The 
brain’s  solution  to  this  problem  appears  to  be  the  use  of  a  perceptual  window  -  events 
(stimuli)  occurring  close  enough  in  physical  time  fall  within  the  perceptual  tolerance  of  the 
perceptual  window.  The  resulting  perception  is  simultaneous  occurrence.  Knowledge  of  the 
existence  of  a  perceptual  window  is  important  from  a  Human  Factors  standpoint  in  regards 
to  multi-sensory  integration.  Specifically,  any  human-machine  interface  implementing  multi- 
sensory  stimulation  must  be  designed  so  as  to  minimize  the  impact  of  one  sense  modality 
upon  another.  Failure  to  do  so  may  result  in  an  unintended  perceptual  awareness  of  incorrect 
stimuli. 

4.  The  existence  of  the  perceptual  window  described  above  gives  rise  to  perhaps  the  most  sig¬ 
nificant  (and  most  exciting)  finding;  noncausality  in  multi-modal  perception.  Experiment  III 
of  Chapter  4  demonstrated  the  ability  of  one  modal  stimulus  to  affect  the  future  perception 
of  another  modal  stimulus.  Such  an  ability  may  allow  the  brain  to  maintain  a  seamless, 
consistent,  and  stable  internal  “world  model”  of  its  external  environment.  The  “price”  for 
this  construction  of  such  a  model:  the  occurrence  of  stimulus  input  illusions...  al  la  apparent 
motion. 
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Appendix  A.  Grossberg‘s  Static  Boundary  Contour  System 
A.l  Background 

Back  in  1984  Grossberg  et  al.  introduced  the  Static  Boundary  Contour  System  (BCS)  model. 
The  model  was  developed  to  simulate  the  visual  system’s  ability  to  detect,  complete,  and  regularize 
boundary  segmentations  in  response  to  a  variety  of  retinal  images(7)  The  model  consists  of  two 
major  subdivisions:  a  non-linear  oriented-sensitive  filter  (OC  filter),  and  a  cooperative-competitive 
feedback  network,  called  the  CC  loop(see  figure  48.  A  brief  description  of  each  division  follows. 


Cooperative-  Competitive  Loop 
(CC  Loop) 


hyper¬ 
complex  1^, 

end-stopped 
complex 


Static  A 

Oriented  complex  (J 

Contrast  - 

Filter  simple  fj  (| 

(SOC  Filter)  V  ^ 

LGN  O 

lateral  geniculate  nucleus 


Figure  48.  Static  Boundary  Contrast  System 
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A. 2  Oriented  Contrast  Filter 

The  first  stage  of  the  Oriented  Contrast  Filter  models  the  shunting  on-center  off-surround 
interactions  at  the  retinal  and  LGN  levels. (figure  49)  As  such,  the  model  is  able  to  account  for 
variable  illumination  and,  at  the  same  time,  amplify  local  contrast  in  an  input  image.  The  outputs 


Figure  49.  Shunting  On-Center  Off-Surround  Interactions 

of  these  model  LGN  cells  are  then  input  to  pairs  of  like-oriented  simple  cells  that  are  sensitive  to 
direction  of  contrast  (stage  2). (7)  This  step  models  the  “orientational  hypercolumns”  discussed  by 
Hubert  and  Weisel(3).  For  this  model,  the  columns  consist  of  oriented  receptive  fields  such  as  the 
ones  shown  in  figure  50.  These  fields,  here-to-for  refered  to  as  simple  cells,  exhibit  an  elongated 
shape  in  order  to  increase  orientational  sensitivity.  They  are  divided  into  positive  and  negative 
activation  areas  along  their  major  axes  of  symmetry (3).  Each  simple  cell  corresponds  to  individual 
points  in  a  scene.  As  such,  they  are  used  to  “store”  the  preferred  orientation  of  each  of  these  points. 
The  simple  cells  described  above  send  their  rectified  ouputs  to  like-oriented  complex  cells.  Since  the 
complex  cells  receive  inputs  from  simple  cells  with  opposite  direction  of  contrast  sensitivity,  they 
are  no  longer  sensitive  to  direction  of  contrast.  Outputs  from  complex  cells  are  gated  by  habituative 
chemicals  and  sent  to  a  corresponding  hypercomplex  cell  network  (referred  to  by  Grossberg  as  the 
first  competitive  stage  of  the  CC  loop).  The  habituation  process  is  necessary  in  order  to  reset 
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Simple  Cells 


Complex  Cell 


Figure  50.  Oriented  Receptive  Fields  -  simple/complex  cells 

boundary  signals  and  thereby  prevent  a  “smearing”  effect  which  would  otherwise  result  during  the 
rapid  presentation  of  input  images  in  time. 

A. 3  Competition  Cooperation  Loop 

First  Competitive  Stage:  Once  again,  the  interactions  between  complex  and  hypercom¬ 
plex  cells  can  be  modeled  as  an  on-center  off-surround  network  whereby  the  “on-center”  cells  are 
activated  while  the  “off-surround”  cells  are  inhibited.  As  a  result  of  this  operation,  hypercomplex 
cells  of  like  position  and  orientation  are  turned  “on”  (excited),  while  at  the  same  time  turning 
“off”  hypercomplex  cells  of  the  same  orientation  but  at  different  locations.  One  way  to  visualize 
this  process  is  to  think  of  the  cell  structure  as  a  volume  in  position  (x,y)  and  orientation  space  in 
which  all  complex  cells  of  a  like  orientation  combine  to  form  a  layer  in  the  complex  cell  volume 
(3).  (See  figure  51).  Imposing  a  center-surround  receptive  field  on  each  complex  cell  at  each  x,y 
position  in  space  excites  the  complex  cell  at  this  position  while  inhibiting  like  oriented  complex 
cells  at  surrounding  positions  within  a  given  orientation  layer.  (3)  The  effect  is  twofold:  the  edges 
detected  by  the  complex  cells  are  strengthened,  and  perpendicular  boundaries  are  formed  at  the 
ends  of  these  like-oriented  response  edges.  Grossberg  describes  this  process  as  end  cutting. 


orientation 

A 


Second  Competitive  Stage:  The  outputs  from  the  first  layer  of  hypercomplex  cells  serve 
as  inputs  for  a  second  round  of  competions  in  a  higher-order  on-center  off-surround  network.  At 
this  level,  different  orientations  compete  to  be  the  dominant  input  at  hypercomplex  cells  corre¬ 
sponding  to  spatial  position.  Recall  that  the  first  competitive  stage  excites  a  range  of  orientations 
(layers)  at  specific  locations(figure  50)  The  second  stage  uses  these  orienations  as  inputs  and  picks 
the  “best”  orientation  for  that  particular  spatial  location.  Once  again,  the  result  of  such  a  com¬ 
petition  is  to  sharpen  existing  boundary  signals.  Cooperation  Stage:  Outputs  from  the  second 
competitive  stage  are  input  into  cooperative  bipole  cells  that  initiate  long-range  boundary  grouping 
and  completions. (7)  The  bipole  cells  construct  boundaries  by  measuring  the  location  and  density 
of  surrounding  hypercomplex  cell  inputs  within  specific  orientation  layers.  If  the  receptive  fields  of 
a  bipole  are  “sufficiently”  activated,  then,  the  bipole  cell  fires.  The  activation  of  individual  bipole 
cells  feeds  back  to  hypercomplex  cells  of  the  same  orientation  and  position  and  inhibits  nearby 


cells.  As  a  result,  spatially  and  orientationally  consistent  boundaries  are  enhanced,  and  incon¬ 
sistent  boundaries  are  inhibited.  An  example  of  this  competition-cooperation  loop  is  presented  in 
figure  52(3)(10).  The  two  active-on  cells  (of  the  same  orientation)  in  the  lower  half  of  the  figure  acti¬ 
vate  the  receptive  fields  of  the  bipole  filter(top  horizontal  line).  This  process  is  represented  by  path 
1.  Having  been  sufficiently  activated,  the  bipole  cell  fires  (path  2)  thereby  exciting  hypercomplex 
cells  within  that  orientation  layer.  The  new  “cell  excitation”  configuration  is  fed  back  through  the 
competitive  stages.  If  these  hypercomplex  excitations  survive,  a  second  bipole  cell  may  fire  (path  3) 
Once  again,  the  dipole  cell  will  excite  corresponding  hypercomplex  cells  within  the  same  orientation 
layer  (path  4).  This  process  of  bottom-up  and  top-down  interaction  between  hypercomplex  cells 
and  bipole  cells  is  repeated  until  the  output  converges  to  a  final  boundary  segmentation. (7). 


A.Ji.  BCS  and  Apparent  Motion: 

As  was  stated  above,  the  simple  cells  of  a  BCS  are  sensitive  to  direction-of-contrast.  However, 
this  sensitivity  is  removed  when  the  outputs  from  simple  cell  pairs  with  opposite  direction-of- 
contrasts  are  input  into  complex  cells.  The  resulting  network  can  therefore  extract  boundary 
structures  independent  of  illumination  fluctuations  (ie.,  shadows)  Unfortunately  this  same  network 
is  made  insensitive  by  these  properties  to  direction-of-motion.  By  itself,  then,  the  existing  BCS  can 
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not  account  for  real  or  apparent  motion  processing.  What  is  needed,  then,  is  a  system  which  can  join 
the  existing  network  to  an  additional  “stage”  of  processing  which  is  sensitive  to  direction-of-motion. 
Grossberg’s  Motion  BCS  performs  this  task. 
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Appendix  B.  Sample  MOC  Filter  Output 


For  the  purposes  of  the  simulations  which  follow,  the  front-end  of  the  MOC  filter  was  simplied 
by  assuming  that  the  transient  cell  responses  always  took  on  a  value  of  1(12).  (This  same  assumption 
was  used  by  Grossberg  so  as  to  simplify  demonstration  of  the  filter’s  output).  In  the  first  section 
which  follows,  the  spatial  separation  between  stimuli  was  varied  from  40  pixels  to  550  pixels  while 
the  temporal  separation  was  held  constant  at  10  ms.  In  the  second  section,  the  spatial  separation 
was  held  constant  at  240  pixels  while  the  temporal  separation  was  increased  from  40  ms  to  440  ms. 


B.l  Spatial  Separation 


Input  >  40  pixel  Spatial  Separation  Level  5  MOC  Output 
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Figure  53.  Spatial  Separation  =  40  pixels 


Input  -  00  pixel  Spatial  Separation  Level  5  MOC  Output 


20  40  60  20  40  60 

tlme(x1 0  ms)  time(x  1 0  ms) 


Figure  54.  Spatial  Separation  =  80  pixels 
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Input  -  120  pixel  Spatial  Separation 


Level  5  MOC  Output 


20  40  60 

time(x10ms) 


20  40  60 

time(x10  ms) 


Figure  55.  Spatial  Separation  =  120  pixels 


Input  -  160  pixel  Spatial  Separation 


20  40  60 

ttme{xl0ms) 


Level  5  MOC  Output 


Figure  56.  Spatial  Separation  =  160  pixels 


Input  -  200  pixel  Spatial  Separation 


20  40  60 

time(x10  ms) 


Level  5  MOC  Output 


20  40  60 

time(xlOms) 


Figure  57.  Spatial  Separation  =  200  pixels 


Input  -  240  pixel  Spatial  Separation 


20  40  60 

time(x10ms) 


Level  5  MOC  Output 


20  40  60 

time(x10  ms) 


Figure  58.  Spatial  Separation  =  240  pixels 
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Input  -  280  pixel  Spatial  Separation 


20  40  60 

timefxIOms) 


Level  5  MOC  Output 


20  40  60 

time(x10ms) 


Figure  59.  Spatial  Separation  =  280  pixels 


Input  -  320  pixel  Spatial  Separation 


20  40  60 

time(xlOms) 


Level  5  MOC  Output 


Figure  60.  Spatial  Separation  =  320  pixels 


Figure  61.  Spatial  Separation  =  360  pixels 


Input  -  400  pixel  Spatial  Separation 


20  40  60 

time(xlOms) 


Level  5  MOC  Output 


20  40  60 

tlme{x10  ms) 


Figure  62.  Spatial  Separation  =  400  pixels 
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Input  -  550  pixel  Spatial  Separation 


20  40  60 

time(xlOms) 


Level  5  MOC  Output 


20  40  60 

time(x10ms) 


Figure  63.  Spatial  Separation  =  550  pixels 


B.2  Temporal  Separation 


Input  -  40  ms  temporal  separation 


20  40  60 

time(xlOms) 


Level  5  MOC  Output 


20  40  60 

time(x10  ms) 


Figure  64.  Temporal  Separation  =  40  ms 


Input  -  80  ms  temporal  separation 


20  40  60 

time(x10  ms) 


Level  5  MOC  Output 


Figure  65.  Temporal  Separation  =  80  ms 


Input  -  1 20  ms  temporal  separation 


20  40  60 

time{x10  ms) 


Level  5  MOC  Output 


20  40  60 

lime(x10  ms) 


Figure  66.  Temporal  Separation  =  120  ms 


Input  -  160  ms  temporal  separation 


20  40  60 

time(xlOms) 


Level  5  MOC  Output 


20  40  60 

time(x10  ms) 


Figure  67.  Temporal  Separation  =  160  ms 


Input  -  200  ms  temporal  separation 


20  40  60 

time(xlOms) 


Level  5  MOC  Output 


20  40  60 

time(x10  ms) 


Figure  68.  Temporal  Separation  =  200  ms 


Figure  69.  Temporal  Separation  =  240  ms 
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Input  -  280  ms  temporal  separation 


20  40  60 

time(x10  ms) 


Level  5  MOC  Output 


20  40  60 

time(x10  ms) 


Figure  70.  Temporal  Separation  =  280  ms 


Input  -  320  ms  temporal  separation 


20  40  60 

time(xl0  ms) 


Level  5  MOC  Output 


20  40  60 

lime(x10  ms) 


Figure  71.  Temporal  Separation  =  320  ms 


Input  -  360  ms  temporal  separation 


20  40  60 

time(x10  ms) 


Level  5  MOC  Output 


20  40  60 

time(x10  ms) 


Figure  72.  Temporal  Separation  =  360  ms 


Input  -  400  ms  temporal  separation 


20  40  60 

time(xlOms) 


Level  5  MOC  Output 


20  40  60 

time(x10ms) 


Figure  73.  Temporal  Separation  =  400  ms 
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Figure  74.  Temporal  Separation  =  440  ms 
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Appendix  C.  Motion  Oriented  Contrast(MOC)  Filter  Computer  Code 

*  * 

*  Grossberg’s  Motion  Oriented  Contrast  Filter  * 

*  (MOC)  * 

*  * 

*  This  routing  implements  Grossberg’s  * 

*  Motion  Oriented  Contrast  Filter  (MOC)  * 

*  Key  parameters  are  identified  in  * 

*  appropriate  subroutines  * 

*  Designer:  Don  Hill,  GE  95D  * 

*  Thesis  Research  * 

*  * 

#include  <stdio.h> 

#include  <math.h> 

#include  <stdlib.h> 

#include  "macros_bcs .h" 

#define  SQR(a)  (a)* (a) 


float  Shunt  0; 


float  Orient  0; 


float  Transient  0; 
float  Transmit  0; 
void  free_matrix() ; 
void  output  0; 

float  **matrix(); 
mainCargc ,argv) 

char  *argv[]; 
int  argc ; 

{ 

int  Row,  Col,  i,  j,  k,  1,  loci,  loc2; 
float  H,  J,  F,  G,  N,  P,  0; 

float  xl_in_DB ,xl_in_BD ; 

float  result ,resultl , total; 

float  **image_array ,  **bcs_array,  **xl_array; 

float  **x2BD_array ,  **x2DB_array ; 

float  **y_array,**dy_array; 

float  * *w_on_ array ,  **w_off _array; 

float  **z_on_array ,  **z_off_array ; 

float  **x3_on_array ,  **x3_off_array ; 
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float 


**BD_on_array ,  **BD_off _array; 
float  **DB_oii_array ,  *>i'DB_off_array ; 

float  **local_r_array ,  **local_l_array ; 

float  **kernel,  **R5_array,  **L5_array; 

float  **R_5_out,  **L_5_out; 
float  ***out_array ,  *in_array; 

FILE  *image_ptr,  *file_ptr,  *FID; 
char  in_string  [80] ; 

/*  read  in  input  file  */ 

ifCargc  !=  4) 

printfC'!!!  The  command  line  should  be:  !!!\n\n  test  [in_file] [row_length] [col_length]  (' 
exit (0) ; 

} 

if ( (image_ptr  =  fopen(argv[l] ,  "r"))  ==  NULL) 

{ 

printfC'I  can’t  find  the  file.  \n"); 
exit (1) ; 

} 
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Row  =  atoi(argv[2] ) ; 
Col  =  atoi(argv[3] ) ; 


/*********  Allocate  memory  for  arrays  *********/ 

image_array  =  matrix(0,  Row-1,  0,  Col-1); 
bcs_array  =  matrix(0,  Row-1,  0,  Col-1); 
xl_array  =  matrix(0,  Row-1,  0,  Col-1); 
x2BD_array  =  matrix (0,  Row-1,  0,  Col-1); 
x2DB_array  =  matrix (0,  Row-1,  0,  Col-1); 
y_array  =  matrix(0,  Row-1,  0,  Col-1); 
dy_array  =  matrix(0,  Row-1,  0,  Col-1); 
w_on_array  =  matrix(0,  Row-1,  0,  Col-1); 
w_off_array  =  matrix (0,  Row-1,  0,  Col-1); 
z_on_array  =  matrix(0,  Row-1,  0,  Col-1); 
z_off_array  =  matrix(0,  Row-1,  0,  Col-1); 
x3_on_array  =  matrix(0,  Row-1,  0,  Col-1); 
x3_off_array  =  matrix (0,  Row-1,  0,  Col-1); 
BD_on_array  =  matrix(0,  Row-1,  0,  Col-1); 
BD_off_array  =  matrix (0,  Row-1,  0,  Col-1); 
DB_on_array  =  matrix(0,  Row-1,  0,  Col-1); 
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DB_off_array  =  matrix(0,  Row-1,  0,  Col-1); 
local_r_array  =  matrix (0,  Row-1,  0,  Col-1); 
local_l_array  =  matrix(0,  Row-1,  0,  Col-1); 
kernel  =  matrix (0,  Row-1,  0,  Col-1); 

R5_array  =  matrix(0,  Row-1,  0,  Col-1); 

L5_array  =  matrix(0,  Row-1,  0,  Col-1); 

R_5_out  =  matrix(0,  Row-1,  0,  Col-1); 

L_5_out  =  matrix (0,  Row-1,  0,  Col-1); 

/****j|cj|t***  Read  in  image  array  ****************/ 

loopij (Row, Col) 

f  scant  (image.ptr,  "°/,f  ",  &image_array  [i]  [j]  )  ; 

Z*********  Read  in  Static  BCS  inputs  **********/ 

FID=f open("/tmp_mnt/home/hawkeye7/95d/dehill/BCS/MDC_code/ simbcs.dat" , "rt") 
loopij (Row, Col) 

fscanf(FID,  "“/.f  ",  &bcs_array  [i]  [j]  )  ; 

/*********  Preprocess  input  (level  1)  *********/ 

printf ("Preprocessing  data  (shimting  response)  \n") ; 
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for  (i=0 ; i<Row; i++) 


result  =  0.0; 
for  (j=0; j<Col; j++) 

{ 

xl_array[i] [j]  =  result; 

result=Shtuit (xl_array [i] [j] ,image_array [i]  [j]) ; 
xl_array[i] [j]=result; 

} 


printf("\n  \n  ******Level  1  Finished*******  \n  \n‘'); 


*«>«***  Create  Oriented  Sustained  Cells  ******/ 
/*********  (Level  2)  ******/ 

/*note:  two  types  of  oriented  sustained  cells  must 
be  computed:  bright-dark  and  dark-bright  cells*/ 
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printf ("creating  Oriented  sustained  cells  (level  2)  \n"); 

for  (i=0;i<Row;i++) 

{ 

result  =  0.0; 
resultl  =  0.0; 

for  (j=0; j<Col; j++) 

{ 

if  (i==0)  /*  special  case  —  edge  of  input  */ 
xl_in_DB  =  xl_array[i]  [j]  ; 
else 

if (xl_array[i]  [j]  -  xl_array [(i-1)]  [j]  >  0) 

xl_in_DB  =  xl_array[i] [j]  -  xl_array [(i-1)] [j] ; 
else 

xl_in_DB  =  0.0; 


if  (i==(Row-l))  /*  special  case  —  edge  of  input  */ 
xl_in_BD  =  xl_array[i] [j] ; 
else 

if (xl_array[i] [j]  -  xl_array[(i+l)] [j]  >  0) 

xl_in_BD  =  xl_array[i] [j]  -  xl_array[(i+l)] [j] ; 
else 
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xl_in_BD  =  0.0; 


x2BD_array[i]  [j]  =  result; 

x2DB_array[i] [j]  =  resultl; 

result=  Orient (x2BD_array[i] [j] ,xl_in_BD,0) ; 

resultl  =  0rient(x2DB_array [i] [j] ,xl_in_DB,0) ; 

x2BD_array [i] [j]=result; 

x2DB_array [i] [j]=resultl; 

> 

} 

printf("\n  \n  ******Level  2  Finished*******  \n  \n"); 


/***>K>X’)<***  Create  Level  3  Transient  Cells  *»<*♦**/ 


printf("\n  \n  calculating  y  array  \n  \n"); 

for  (i=0 ; i<Row; i++) 

{ 

result=0; 

for  (j=0; j<Col; j++) 

{ 

y_array[i]  [j]  =  result; 
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result  =  Transient (y_array[i] [j] ,xl_array[i] [j]  ,0) ; 


y_array[i]  [j]  =  result; 


/♦printf  ("xl:  '/.f  y:  °/,f  \n"  ,xl_array  [i]  [j]  ,y_array  [i]  [j]  ) ;  */ 

} 


} 


printf("\n  \n  ******  y_array  calculated  ********  \n  \n"); 


printf("\n  \n  ******  calculating  dy/dt  *******  \n  \n"): 


F  =  0.4; 
G  =  2.0; 


for  (i=0; i<Row; i++) 

{ 

for  (j=0; j<Col; j++) 

{ 

/*dy_array[i]  [j]  =  -F*y_array [i]  [j]  +  (G-y_array [i]  [j] )  *  (0  +  xl.array [i] [j] ) ; */ 


if  (j==0) 


{ 

dy_array[i]  [j]  =  ( (y_array [i] [j+1] -y_array [i] [j] )/ . l)/2; 
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} 


else 

if  (j==(Col-l)) 

{ 

dy_array [i] [j]  =  ( (y_array [i] [j] -y_array [i] [j-1] )/ . 1) ; 

} 

else 

{ 

dy_array[i]  [j]  =  ( ( (y.array  [i]  [j+1] -y.array  [i]  [j] )/.  l)  +  ((y_array[i]  [j]-y_array[i]  [j-1] 

> 

> 

} 

printf("\n  \n  ******  generating  transient  half  wave  rectifications  ******  \n  \n"); 

H  =  0.5; 

J  =  -0.001; 

for  (i=0 ; i<Row; i++) 

{ 
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for  (j=0; j<Col; j++) 

{ 

w_on_array [i] [j]  =  dy_array[i] [j]  -  H; 
if  (w_on_array [i] [j]  <  0) 
w_oii_ array [i]  [j]  =0; 

w_off_array[i]  [j]  =  J  -  dy_array [i]  [j] ; 
if  (u_off _array [i] [j]  <  0) 
w_off _array [i] [j]  =  0; 

} 

> 

printf("\n  \n  ******  w  arrays  calculated  ******  \n  \n"); 

printf("\n  \n  ******  calculating  transmitter  gate  array,  z  ******  \n  \n") 

for  (i=0;i<Row;i++) 

{ 

result  =  0.0; 
result 1  =  0.0; 

for  (j=0; j<Col; j++) 
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{ 

z_on_array[i] [j]  =  result; 
z_off_array[i]  [j]  =  resultl; 

result=Transmit  (z_on_array  [i]  [j]  , w_oii_ array  [i]  [j]  ) ; 
resultl=Transinit(z_off_array[i]  [j]  ,w_off _array [i]  [j]) ; 
z_oii_array[i]  [j]  =  result; 
z_off_array[i] [j]  =  resultl; 

} 

} 

printf("\n  \n  ******  z_arrays  calculated  ********  \n  \n"); 

printf("\n  \n  ******  calculating  level  3  trcinsient  response  ******  \n  \n" 

for  (i=0; i<Row; i++) 

for  (j=0; j<Col; j++) 

{ 

x3_on_array[i] [j]  =  w_on_array [i] [j] *z_on_array [i] [j] ; 
x3_off_array[i] [j]  =  w_off_array [i] [j]*z_off_array[i] [j] ; 

} 
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printf("\n  \n  ******  level  3  transient  response  cells  calculated  ******  \n  \n"); 

printf("\n  \n  ******Level  3  Finished*******  \n  \n"); 

/******  Level  4  Processing  ******/ 

printf("\n  \n  ******  calculating  sustained  transient  cells  ******  \n  \n"); 

for  (i=0 ; i<Row; i++) 

{ 

for  (j=0; j<Col; j++) 

{ 

BD_on_array[i]  [j]  =  x2BD_array[i]  [j]  *  x3_on_array[i]  [j]  ; 

BD_off_array[i] [j]  =  x2BD_array[i] [j]  *  x3_off .array [i] [j] ; 

DB_on_array[i]  [j]  =  x2DB_array[i]  [j]  *  x3_on_array[i]  [j]  ; 

DB_off_array[i] [j]  =  x2DB_array[i]  [j]  *  x3_off_array[i]  [j] ; 

} 

} 

printf("\n  \n  ******Level  4  Finished*******  \n  \n"); 


/******  Level  5  Processing  ******/ 


printf("\n  \n  *****  calculating  local  and  global  motion  cells  *****  \n  \n"); 


N  =  5.0; 

P  =  1.0; 

for  (i=0 ; i<Row; i++) 

{ 

for  (j=0; j<Col; j++) 

{ 

/*local_r_array[i]  [j]  =  N  *  BD_on_array[i] [j]  +  P  *  DB_off_array[i]  [j] ;*/ 
local_l_array[i]  [j]  =  P  *  BD_off_array[i] [j]  +  N  *  DB_on_ array [i]  [j] ; 

/*  level  5  input  set  equal  to  level  1  sustained  cells  if  transient  response 
assumed  to  be  1:  */ 

local_r_array [i] [j]  =  xl_array [i]  [j] ; 

} 

> 

printf("\n  \n  caculating  Gaussian  kernal  \n  \n  "); 
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Q  =  60.0;  /*  sets  breadth  of  kernel  *! 

for  (i=0; i<Row; i++) 

{ 

for  (k=0;k<Row;k++) 

kernel [i] [k] =exp(-SqR(k-i)/ (2  *  SQR(Q))); 

} 

} 

for  (i=0;  i<Row;i++) 

{ 

for  (j=0; j<Col; j++) 

■C 

R5_array[i]  [j]  =  0; 

L5_array[i]  [j]  =  0; 
for  (k=0;k<Row;k++) 

{ 

R5_array[i] [j]  =  R5_array[i] [j]  +  kernel [i] [k] *local_r_array[k]  [j] ; 
L5_array[i]  [j]  =  L5_array[i]  [j]  +  kernel  [i]  [k]  >Klocal_l_array[k]  [j]  ; 

« 

} 

} 
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for  (i=0;i<Row;i++) 


for  (j=0; j<Col; j++) 

{ 

R_5_out [i] [j] =0; 
L_5_out [i] [j]=0; 

> 


for  (j=0; j<Col; j++) 

{ 

result  =  R5_array[0]  [j]  ; 
loci  =  0; 
loc2  =  j ; 

for  (i=l ; i<Row; i++) 

{ 

if  (R5_array[i]  [j]  >  result) 

{ 

result  =  R5_array[i] [j] ; 
loci  =  i; 
loc2  =  j ; 

> 

> 

R_5_out [loci]  [loc2]  =  R5_array [loci]  [loc2] 

} 
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for  (j=0; j<Col; j++) 

{ 

result  =  L5_array[0] [j] ; 
loci  =  0; 
loc2  =  j ; 

for  (i=l ; i<Row; i++) 

{ 

if  (L5_array[i] [j]  >  result) 

{ 

result  =  L5_array[i]  [j] ; 
loci  =  i; 
loc2  =  j ; 

} 

} 

L_5_out [loci]  [loc2]  =  L5_array [loci]  [loc2] ; 

} 

output (R5_array, Row, Col) ; 


/♦*♦***  Free  up  allocated  memory  *♦***♦*/ 
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free_matrix(image_array ,0,Row-l ,0,Col-l) ; 
f ree_matrix (bcs_array , 0 , Row-1 , 0 , Col-1) ; 
f ree_matrix (xl_array , 0 , Row-1 , 0 , Col-1 ) ; 
free_matrix(x2BD_array,0, Row-1 ,0, Col-1) ; 
f ree_matrix (x2DB_array , 0 , Row-1 , 0 , Col-1) ; 
f ree_matrix (y_array , 0 , Row-1 , 0 , Col- 1 ) ; 
f ree_matrix (dy_array , 0 , Row-1 , 0 , Col-1 ) ; 
f ree_matrix(w_on_ array ,0, Row-1 ,0, Col-1)  ; 
free_matrix(w_off_array,0, Row-1 ,0, Col-1) ; 
f ree_matrix(z_on_ array ,0, Row-1 ,0, Col-1) ; 
f ree_matrix(z_off .array ,0,Row-l ,0,Col-l) ; 
f ree_matrix(x3_on_ array, 0, Row-1 ,0, Col-1) ; 
f ree.matrix (x3_off .array , 0 , Row-1 , 0 , Col-1 ) ; 
free.matrix(BD_on.array ,0 ,Row-l ,0 ,Col-l) ; 
free.matrix(BD.off .array ,0 ,Row-l ,0, Col-1) ; 
f ree.matrix (DB.on.array , 0 , Row- 1,0, Col-1) ; 
f ree.matrix (DB.off. array , 0 , Row-1 , 0 , Col-1) ; 
f ree.matrix (local.r.array , 0 , Row-1 , 0 , Col-1 ) ; 
free.matrixdocal. 1. array, 0, Row-1 ,0, Col-1)  ; 
free.matrix(kernel, 0, Row-1 ,0, Col-1) ; 
f ree.matrix (R5. array , 0 , Row-1 , 0 , Col-1) ; 
f ree.matrix (L5. array , 0 ,Row-l , 0 , Col-1)  ; 
free.matrix(R_5_out ,0 ,Row-l ,0 ,Col-l) ; 
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free_matrix(L_5_out ,0,Row-l ,0,Col-l) ; 

> 

/*********  End  of  Main  Program  ******♦**♦♦/ 


/**♦******  Begin  Subroutines  *************/ 


/*4t*******  Level  1  Shunting  Response  ***** 

t  =  initial  time,  tO 
h  =  range  of  differentiation 
n  =  number  of  iterations 
X  =  integration  variable 
input  =  image  input 
A  =  rate  of  passive  decay 
B  =  maximiim  activity  of  cell 

This  subroutine  performs  an  integration  to 
construct  the  shunting  response  of  input 
image 
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float  Shunt (x, input) 

float  x; 
float  input ; 

{ 

float  t=0; 
float  h=. 1 ; 
float  n=100; 
float  FNF; 
float  A=.5; 
float  B=10; 

h=(h-t)/n; 

while (n>0) 

{ 

FNF  =  -.A*x  +  (B-x)*input; 
x=x+h*FNF ; 
t=t+h; 
n=n-l ; 
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> 


return(x) ; 

} 


/********  Level  2  Oriented  Sustained  Cells  **** 

two  matrices  —  x2BD,  x2DB  use  Orient  to  produce 
oriented  sustained  cells 

t  =  initial  time,  tO 
h  =  range  of  differentiation 
n  =  number  of  iterations 
X  =  integration  variable 
input  =  image  input 
bcs  =  bcs  input 
C  =  rate  of  passive  decay 
D  =  maximum  activity  of  cell 
E  =  oriented  input  scale  value 

This  subroutine  performs  an  integration  to 
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construct  the  oriented  sustained  response  for 


bright-dark  and  dark-bright  cells 

float  Orient (x, input ,bcs) 

float  x; 
float  input ; 
float  bcs; 

{ 

float  C=0.1; 
float  D=10.0; 
float  E=10.0; 

float  t=0; 
float  h=.l; 
float  n=100; 
float  FNF; 

h=(h-t)/n; 
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while (n>0) 


{ 

FNF  =  -C*x  +  (D-x)  *  E  *  (bcs  +  input) ; 
x=x+h*FNF ; 
t=t+h; 
n=n-l ; 

> 

return(x) ; 

} 

Level  3  Transient  Cells  ***** 

t  =  initial  time,  tO 

h  =  range  of  differentiation 

n  =  number  of  iterations 

y  =  integration  variable 

input  =  level  1  sustained  response  input 

bcs  =  bcs  input 

F  =  rate  of  passive  decay 

G  =  maximum  activity  of  cell 

This  subroutine  performs  an  integration  to 
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calculate  transient-on  and  transient-off 


response  activity 

float  Transient (y, input, bcs) 

float  y; 
float  input ; 
float  bcs; 

{ 

float  F=0.4; 
float  G=2.0; 

float  t=0; 
float  h=.l; 
float  n=100; 
float  FNF; 

h=(h-t)/n; 
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while (n>0) 


{ 

FNF  =  -F*y  +  (G-y)*(bcs  +  input); 
y=y+h*FNF ; 
t=t+h; 
n=n- 1 ; 

} 

return(y) ; 

} 

/*******>(<♦  Level  3  Transmitter  Gates  ***** 

t  =  initial  time,  tO 
h  =  range  of  differentiation 
n  =  number  of  iterations 
z  =  integration  variable 

input  =  rectified  transient  response  input 
K  =  accumulation  rate 
L  =  maximum  activity  of  cell 
M  =  habituation  scale  parameter 

This  subroutine  performs  a  habituation  of 
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the  rectified  transient  response  inputs 


float  Transmit (z , input) 

float  z; 
float  input ; 

{ 

float  K=0.06; 
float  L=3.0; 
float  M=5.0; 

float  t=0; 
float  h=.l; 
float  n=100; 
float  FNF; 

h=(h-t)/n; 

while (n>0) 

{ 
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FNF  =  K*(L-z)  -  M*input*z; 


z=z+h*FNF ; 


t=t+h; 
ii=n-l ; 
> 


return(z) ; 

} 


/♦♦♦♦♦♦Jit  Function  output  *******/ 

void  output (in_arr ay, Row, Col) 
float  *=i‘in_ array ; 
int  Row; 
int  Col; 

{ 

int  i,j,k; 

FILE  *file_ptr; 

printf ("\n\n") ; 

printfC'***  Making  the  output  file  ***") 
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printf ("\n\n") ; 


file_ptr  =  fopen("R5arb.dat" ,  "w"); 


for  (i=0;i<Row;i++) 

{ 

for  (j=0; j<Col; j++) 

{ 

fprintf (f ile_ptr ,  "Xf  ",  in_array [i]  [j] ) ; 

} 

fprintf (file_ptr,  "\n"); 

> 


printf  ("  ****>«**  DONE  WITH  BCS  *******"'); 

printf ("\n\n") ; 


> 
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Appendix  D.  PCNN  Level  2  Apparent  Motion  Pulse  Output 
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Appendix  E.  Pulse  Coupled  Neural  Network  Code  (Matlab  Implementation) 

y.  Pulse  coded  neural  network 

%  Written  by  Capt  Randy  Broussard.  Modified  by  Capt  Don  Hill 

clear  all 

LRadiusl=0;  %  radius  of  linking 
Beta=0;  '/,  linking  strength  (was  -.125) 

Tl=5;  °/o(10)  Time  constant  of  U  pulse  (Input  +  Linking  signals) 

Vl=5;  "/,  Amplification  of  U  signal  (U  pulse  height) 

Vf=5; 

w=l;  y,  all  weights  set  to  1  for  this  case 

Thetal_0=.l;  y,(.l)  threshold,  sets  lowest  input  that  can  create  a  pulse 
Theta2_0=10;  '/,  level  2  threshold 

Ts=.5;  y.  Time  constant  of  inhibiting  pulse  (Theta) 

Vs=10;  y,  Amplif .  of  pulse  inhibiting  signal  (Theta  pulse  height) 

FRadius2=0;  7,  (was  5)  Feeding  radius  of  linking  in  layer  2 
LRadius2=0;  7»  (was  5)  Linking  radius  of  linking  in  layer  2 
MaxRadius=max (FRadius2 , LRadius2) ; 

qF=4;  7o  Spatial  Kernel  weight  (larger  Q,  broader  kernel) 

qL=2; 

Beta2=  -.015;  ’/,  -.125;  linking  strength  in  layer  2  (was  -.125) 
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%Tfd=l;  %  Decay  time  constant  of  F  pulse  (Input  signal) 
°/,Tfr=l;  7o  Rise  time  constant  of  F  pulse  (Input  signal) 
°/oVf=25;  7o  Amplif.  of  input  pulse  signal 

wF2=zeros ( (2*FRadius2+l) , (2*FRadius2+l) ) ; 
wL2=zeros ( (2*LRadius2+l) , (2*LRadius2+l) ) ; 

for  i=  1 : (2*FRadius2+l) 
for  j=  1 : (2*FRadius2+l) 
if  (i==(FRadius2+l) ) 

wF2(i,  j)=exp(-((abs(i-j))'‘2)/(2*QF~2))  ; 
else 

wF2(i,j)=0; 

end; 

end; 

end; 

wF2 

for  i=  1 : (2*LRadius2+l) 
for  j=  1 : (2*LRadius2+l) 
if  (i==(LRadius2+l) ) 
wL2(i, j)=exp(-((abs(i-j)) "2)/(2*QL~2)) ; 
else 
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wL2(i, j)=0; 


end; 

end; 

end; 

wL2 

Tl=exp(-1/Ts) ; 

T2=exp(-1/Tl) ; 
load  box0.dat; 
load  boxl.dat; 

'/.load  box2 .  dat ; 
load  box3 . dat ; 

y,  Input=boxO;  X/DataMax; 
load  box5 . dat ; 

Input=box5 ; 

'/oDataMax=max(max(boxl) ) ; 

%Input=boxl/DataMax ; 
y,load  pcnndata.dat; 
yoDatELMax=max(max(pcnndata) ) ; 

y,Input=pcnndata/DataMax;  ’/.image  must  be  padded  for  correctness 
[Ysize,Xsize]=size(Input) ; 

°/,Theta=Input>0 ; 

’/,X=Theta; 

Theta=zeros (Xsize , Ysize) ; 
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X=zeros (Xsize , Ysize) ; 


L=zeros (Xsize , Ysize) ; 
F=zeros (Xsize , Ysize) ; 
Fl=zeros (Xsize , Ysize) ; 

Y=zeros (Xsize , Ysize) ; 
L2=zeros (Xsize , Ysize) ; 
F2=zeros(Xsize, Ysize) ; 
Theta2=zeros(Xsi2e , Ysize) ; 
figure (1) ; 
clg; 

“/.subplot  (1,2,1)  ; 

“/.spy (Input) ; 
figure (2) ; 

clg; 

“/.figure  (3) ; 

'/.clg; 

“/.figure (4) ; 

'/.clg; 

pause 

for  n=l:25; 
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if  (n==l)  '/opresent  first  box  at  n=l 
yoDataMax=max(max(boxl) ) ; 

Input=box5 ; 
yoTheta=Input>0 ; 
yj=Theta; 
y,X=Input>0 ; 

end; 

y  if  (n==25)  "/opresent  second  box  at  n=50 
yoDataMax=max(max(box2) ) ; 
y,Input=box2/DataMax ; 
y,  Input=boxl ; 

y.Theta=Input>0 ; 
y.X=Theta; 
y,  end ; 

I  if  (n==50) 
y,  Input=boxO ; 

y.  end; 

Theta01d=Theta ; 

y,  fprintf  (1 , ’max  F2=yof  at  begin  of  n=y,d\n’ ,max(max(F2) )  ,n)  ; 

Theta01d2=Theta2 ; 

change=l;  Used  to  tell  when  linking  oscilations  have  ended 
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if (n==l) 


Redraw=l;  %  Redraw  screen  to  clear  drawing  of  time  0  pulse 
else 

Redraw=0;  %  Used  to  tell  if  screen  needs  redrawing 
end 

while (change >0) ; 
change=0 ; 

for  row=l+LRadiusl :  Ysize-LRadiusl  ‘/.loops  that  calculate  the  k-th  neuron 
for  col=l+LRadiusl  :Xsize-LRadiusl  ‘/,k-th  neuron  is  at  location  row, col  in  matrix 
Lk=0; 

for  or=(-LRadiusl)  :LRadiusl  ‘/.loops  that  sum  the  linking  inputs 
for  oc=(-LRadiusl) : LRadiusl 
if(or"=0)  &  (oc~=0) 

Lk=Lk+L(row+or,col+oc) ; 

end 

end 

end 

Fk=Input  (row, col)  ;  ‘/.normally  would  be  sum  of  Fik’s 
‘/.Fk=Fl (row, col)  ; 

Uk=Fk*(l+Beta*Lk) ; 

Theta(row, col)=Thetal_0+Theta01d(row,col)*Tl+Vs*X(row, col) ; 
if (Uk>=Theta(row, col) ) 
if (X(row,col)<l) 
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X(row, col)=l ; 


change=chaiige+l ; 

Redraw=l ; 

end 

end 

°/,>tc>^*!)c***>k*«*******!X*****  debug  probes  ****************************** 
if  (row==12) 
if (col==12) 

ThAl(n)=Theta(row,col) ; 

UAl(n)=Uk; 

end 

y.  if(col==13) 
y,  ThBl(n)=Theta(row,col) ; 
y,  UBl(n)=Uk; 
y,  end 

y.  if(col==10) 

y,  ThCl  (n)=Theta(row, col) ; 

y.  UCl(n)=Uk; 

%  end 
end 

%  fprintf  (1 , ’t=ycd  change=yod  Y(y,d,y,d)=y,d\n' ,t , change ,row, col ,Y(row, col) ) ; 
%****>(c****j(t!(t!(c***!(c>(c***JitJK*  end  debug  probes  *********!<=***************** 
end  y,col 
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end  7orow 


end  ’/.while  changes 
L=L*T2+V1*X;  ’/.weight  is  1 
’/.F1=(F1*T2)+Vf*lnput;  ’/.weight  is  1 

change=l ; 
while (change>0) ; 
change=0 ; 

for  row=(l+MaxRadius)  :  (Ysize-MaxRadius)  ’/.loops  that  calculate  the  k-th  neuron 
for  col=(l+MaxRadius)  :  (Xsize-MaxRadius)  ’/.k-th  neuron  is  at  location  row, col  in  matrix 
Lk=0; 

for  or=(-LRadius2)  :LRadius2  ’/.loops  that  sum  the  linking  inputs 
for  oc=(-LRadius2) :LRadius2 
if(or"=0)  I  (oc~=0) 

Lk=Lk+L2 (row+or , col+oc) *wL2 (or+LRadius2+l , oc+LRadius2+l) ; 

end 

end 

end 

Fk=0; 

for  or=(-FRadius2)  :FRadius2  ’/.loops  that  sum  the  linking  inputs 
for  oc=(-FRadius2) :FRadius2 

Fk=Fk+F2 (row+or , col+oc) *wF2 (or+FRadius2+l ,oc+FRadius2+l) ; 
end  ’/.oc 


118 


end  °/,or 

Uk=Fk* ( 1+ (Beta2*Lk) ) ; 

%Uk=Fk; 

Theta2  (row ,  col)  =Theta2_0+Thet a01d2 (row , col)  =i'Tl+Vs*Y (row ,  col)  ; 

°/,Theta2  (row ,  col)  =Theta2_0+  (Theta01d2  (row , col)  +  ( .  05*Lk) )  *Tl+Vs*Y (row ,  col) ; 
if (Uk>=Theta2 (row , col) ) ; 
if (Y(row,col)<l) 

Y  (row, col)=l ; 
change = change +1 ; 

Redraw=l ; 

end 

end 

y, ********************!(<**  debug  probes  ********>)<*****>i<>(<****>it*****>i<*** 
if  (row==12) 
if (col==12) 

ThA2(n)=Theta2(row,col) ; 

UA2(n)=Uk; 

end 

I  if(col==13) 

y,  ThB2(n)=Theta2(row,  col) ; 
y,  UB2(n)=Uk; 

'/o  end 

y.  if(col==10) 
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°/o  ThC2(n)=Theta2(row,  col) ; 


7.  UC2(n)=Uk; 

7o  end 
end 

"I,  fprintf  (1 , ’t=y,d  change=7od  Y(y.d,y,d)=yod\n’ ,t ,  change , row, col ,Y(row, col) ) 

end  debUg  probeS  ♦♦♦♦♦♦♦♦lit*****  ♦  ♦♦♦jK******** 

end  y.col 

end  y,row2 
end  y, while  changes 
L2=(L2*T2)+V1*Y;  ^weight  is  1 
F2=(F2*T2)+Vf*X;  “/weight  is  1 

/,*******j(!*i(t!)<****j(!j(t******  Update  graphs  as  necessary  ***************** 
fprintf  (1,  ’n=“/,d\n’  ,n)  ; 
if (rein(n,l)==0)  |  (Redraw>0) 
figure  (2) ; 
subplot (2, 1,1)  ; 

plot(l:n,ThAl,  >r-M:n,UAl,’g-’); 

subplot (2, 1,2) ; 

plot(l:n,ThA2, ’r-> ,l:n,UA2, ’g-’) ; 

“/.figure  (2)  ; 

“/.subplot (3, 2, 3) ; 

“/.plot(l:n,ThBl,  ’r-’  ,l:n,UBl,  >g-’); 
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’/osubplot  (3,2,4); 

y,plot(l:n,ThB2,  ’r- >  ,  1  :n,UB2 , 'g-’ )  ; 

'/.figure  (3) ; 

'/.subplot  (3,2,5); 

'/.plot(l:n,ThCl,  ’r-’  ,l:n,UCl,  ’g-’)  ; 

'/.subplot  (3,2,6); 

'/.plot(l:ii,ThC2,  ’r->  ,l:n,UC2,’g-’); 

end 

if (Redraw>0) 

'/.if  (Redraw>2) 
figure(l) ; 
subplot (2,1,1); 
spy(X) ; 

title (’Pulse  Output’); 

'/.figure  (1) 
subplot (2, 1,2) ; 
spy(Y) ; 

'/.title ( ’Pulse  Output  -  Layer  2’); 

fprintf  (1 , ’Pattern  changed  at  time  ‘/,d\n’ ,n) ; 
TitleString=sprintf  (’Pulse'/.d.daf  ,n)  ; 
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title (Titlestring) ; 

FID=f open(TitleString, ’ w’ ) ; 
for  row=l+MaxRadius : Ysize-MaxRadius 
for  col=l+MaxRadius : Xsize-MaxRadius 
fprintf  (FID,  '"/.f  ’  ,Y (row, col))  ; 
end 

fprintf (FID , ’ \n  0 ; 
end 

f close(FID) ; 
pause (1) ; 

X=zeros (Ysize ,Xsize) ; 

Y=zeros (Ysize ,Xsize) ; 
figure (1) ; 
subplot (2,1,1) ; 
spy(X) ; 

subplot (2,1,2); 
spy(Y) ; 

end  '/oif  Redraw 
end  Xn 
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Appendix  F.  Multi-Modal  Test  Set-up  Verification 

Measurements  for  test  set-up  verification  were  carried  out  using  a  dual  channel  Hewlett 
Packard  7000  series  storage  oscilloscope.  The  output  from  the  soundcard’s  speaker/headphone 
jack  provided  the  input  signal  for  channel  1  while  the  voltage  generated  from  a  photocell  directed 
at  the  video  screen  provided  input  to  channel  2.  A  graphical  representation  of  this  test  setup  is 
represented  in  figure  75.  Prior  to  test  verification,  the  size  of  the  visual  stimuli’s  black  boxes  was 


Oscilloscope 


Figure  75.  Time  Delay  Verification  Setup 

increased  to  20x20  pixels  to  facilitate  creation  of  a  clean,  easily  recognized  voltage  level  on  the 
oscilloscope.  Under  normal  test  conditions,  a  selection  of  “go”  would  result  in  a  call  to  erase  the 
screen  (except  for  the  centered  black  cross-hair),  followed  shortly  thereafter  by  the  appearence  of 
the  first  of  the  two  boxes.  Using  the  presence  of  one  of  the  black  boxes  as  a  resting  condition, 
the  call  to  erase  it  from  the  screen  resulted  in  a  momentary  rise  in  photocell  voltage  -  until  the 
box  reappeared.  Ideally,  the  oscilloscope  would  have  been  set  to  trigger  on  a  negative  slope  at  a 
level  corresponding  to  the  decrease  in  voltage  caused  by  the  appearence  of  the  black  box.  However, 
illuminance  of  the  video  screen  was  not  constant  -  the  screen  was  refreshed  every  20ms  (50Hz 
refresh  rate).  As  a  result,  the  voltage  seen  by  the  photocell  increased  and  decreased  periodically. 
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To  “capture”  the  temporal  delay  between  onset  of  visual  and  audio  stimuli,  the  oscilloscope  was 
set  to  trigger  off  a  rising  slope  of  channel  2  (photocell  voltage)  at  a  level  corresponding  to  the 
increased  illumination  seen  by  the  photocell  when  the  black  box  was  erased.  By  maximizing  the 
persistence  of  the  oscilloscope  for  both  channels  1  and  2,  the  temporal  delay  between  visual  and 
auditory  stimuli  could  be  measured  and  recorded.  This  procedure  was  repeated  at  10  ms  intervals 
for  a  range  of  delays  between  0  and  40  ms.  Since  the  timing  delays  generated  by  the  hardware 
could  not  be  totally  insulated  from  the  effects  of  the  hardware’s  clock  pulses,  several  measurements 
(30)  were  taken  at  each  interval.  The  recorded  measurements,  as  well  as  their  corresponding  mean 
and  variance,  are  shown  in  table  28.  The  results  of  a  pair-wise  test  of  significance  performed  on 
the  measured  data  is  presented  in  table  29. 
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Table  28.  Test  Setup  Timing  Verification  Results 
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Appendix  G.  MultiModal  Noncausality  Computer  Code 


G.l  Source  (*.CPP)  Code 
ttdefine  WIN31  1 
#include  <windows.h> 

#include  <iimisystem.h> 
ttinclude  <stdio.h> 

#include  <stdlib.h> 

#iiiclude  <math.h> 

HWND  HWindow;  //  handle  of  main  window 

HANDLE  ghinstance;  //  instance  niimber  of  main  window 

HWND  Buttonl,  Button2,  Buttons,  Button4,  Buttons ;  //  Handle  to  buttons  in  main  window 

int  Rxl=260,Ry 1=200, Rx2,Ry2;  //  rectangle  corner  points 

int  Rwidth=5,  Rheight=5;  //  width  and  height  of  rectangle 

int  dx=500;  //  stepsize  for  apparent  motion  dot 

int  Box0nTime=50 ;  //  in  milliseconds 

int  BetweenBoxTime=50;  //  in  milliseconds 

int  SoundDelayTime=100 ;  //  in  milliseconds 

int  MovedRight=0 ;  //  #  times  visual  stimuli  appeared  right  first 
int  MovedLeft=0;  //  #  times  visual  stimuli  appeared  left  first 
int  GuessedRight=0 ;  //  #  times  test  subject  guessed  right 
int  GuessedLeft=0;  //  #  times  test  subject  guessed  left 
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int  TestCount=0;  //  total  #  of  test  trials  completed 

int  SilentCoTint=0;  //  #  of  silent  test  trials  completed 

int  SyncCount=0;  //  #  of  in-order  test  trials  completed 

int  0utofSyncCount=0;  //  #  of  out-of-order  test  trials  completed 

int  Correct=0;  //  total  #  of  correct  choices  made 

int  lncorrect=0;  //  total  #  of  incorrect  choices  made 

int  SilentCorrect=0;  //  #  of  correct  silent  test  choices 

int  SilentIncorrect=0;  //  #  of  incorrect  silent  test  choices 

int  SyncCorrect=0;  //  #  of  correct  in-order  test  choices 

int  SyncIncorrect=0;  //  #  of  incorrect  in-order  test  choices 

int  0utofSyncCorrect=0;  //  #  of  correct  out-of-order  test  choices 

int  0utofSyncIncorrect=0;  //  #  of  incorrect  out-of-order  test  choice 

int  LRdir=0;  //  if  1,  initial  visual  onset  occurs  left 

int  RLdir=0;  //  if  1,  initial  visual  onset  occurs  right 

int  RLsound=0;  //  if  1,  initial  audio  onset  occurs  left 

int  LRsound=0;  //  if  1,  initial  audio  onset  occurs  right 

int  Nosoimd=0;  //  if  1,  indicates  wavefile  contained  no  sound 

int  rnum;  //  dummy  variable  (used  in  case  statements) 

int  flag=0;  //  dummy  variable  (used  in  randomization  call) 

long  LoopsPerMilliSec ;  //  indicates  #  iterative  loops  executed/ms 

long  Loopsl;  //  dummy  variable  (used  in  time  delay  loop) 

long  delta=0;  //  temporary 

long  deltal=0;  //  temporary 
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WORD  devicelD;  //  windows  device  id 

WORD  waveIDH,  wavelDL,  wavelDN;  //  windows  wave  file  id 

//  sound  functions  (these  functions  were  down-loaded  from  a  BBS  on 
//  the  Net) 

/**j(tj|Cj|Cj|(****J|t>|t*****>(<*lt!********=t‘****J|<>(‘**!tt***>tt**j|t**>|C****!(CJ|C***>tt*********** 

FUNCTION  :  ErrorProc (WORD) 

PURPOSE  :  ErrorProc  calls  mciGetErrorString  to  display  an  error 
message  returned  by  MCI . 

COMMENTS  :  N/A 
HISTORY  :  N/A 

void  FAR  PASCAL  ErrorProc (dwResult) 

DWORD  dwResult; 

{ 

HANDLE  hMem; 

LPSTR  IpStringBuf f ; 

hMem=GlobalAlloc(GHND,80) ; 
if  (hMem) 

lpStringBuff=GlobalLock(hMem) ; 
if  (IpStringBuf f) 
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{ 

if (mciGetErrorStringCdwResult  jlpStringBuff ,80) )  MessageBox(NULL,lpStringBuf f , "ERROR" ,MB_OK 
else 

MessageBox(NULL, "Generic  Error" , "ERROR" ,MB_0K) ; 

GlobalUnlock(hMem) ; 

} 

else 

MessageBoxCNULL, "Lock  Failed" , "ERROR" ,MB_0K) ; 

GlobalFree(hMem) ; 

} 

else 

MessageBoxCNULL, "Alloc  Failed" , "ERROR" ,MB_0K) ; 
return; 

} 

/  9|c  3tc  sK  s|c  ^  ^  9fc  :|c)|e  }|e  :4e  :4c  :4c:4c  9|c)(e  )|C  ife  %  94c  94(  9tc  sfc  :4c  :4c  9(C)((  >((*%  ^  *  * 

FUNCTION  :  CloseAllDevices () 

PURPOSE  :  This  function  closes  all  currently  open  devices  for  the  application. 

COMMENTS  :  N/A 
HISTORY  :  N/A 

i|c**>^!tt:4:*l|cs)c:|(>k*:t<********i|<**>l‘*>l<>K>l‘***>)<**i^**********>l<*>l<***!<c!<i!|c>|c!(<:|ci|ts|cs|t********/ 

BOOL  FAR  PASCAL  CloseAllDevices () 
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{ 

DWORD  dwRes; 

//  Close  the  global  device 

dwRes=mciSendConmand(MCI_ALL_DEVICE_ID,MCI_CLOSE,0,NULL) ; 
if  (dwRes) 

{ 

ErrorProc (dwRes) ; 
return (FALSE) ; 

} 

return (TRUE) ; 

} 

/  *****>(<*=(<*****  *★*>(<*****>(<******  ♦♦♦♦♦♦♦■I'********  St'*********************** 


FUNCTION  :  SndPlaySnd(HWND,LPSTR) 

PURPOSE  :  This  function  demonstrates  how  to  use  the  function 

sndPlaySound .  This  is  the  easiest  way  to  play  a  wave  file. 

COMMENTS  :  The  one  setback  to  using  sndPlaySound  is  that  the  entire 

sound  is  loaded  into  memory  at  once .  If  you  have  a  very 
large  sound  sndPlaySoimd  will  fail  if  the  sound  will  not 
load  into  physical  memory. 

HISTORY  :  N/A 
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Jit*********************************************************************/ 


BOOL  FAR  PASCAL  SndPlaySnd(lpWavef ile) 

LPSTR  IpWavefile; 

{ 

WORD  wFlags; 

MessageBox(NULL, IpWavefile, (LPSTR)"Playing  Sound  File" ,MB_0K) ; 
wFlags=SND_ASYNC; 

if  (sndPlaySound(lpWavef ile  jWFlags)) 
return (TRUE) ; 
else 

return (FALSE) ; 

} 

FUNCTION  :  OpenWaveDevice (HWND) 

PURPOSE  :  This  function  opens  the  device  waveaudio  and  leaves  it  open. 

The  device  could  also  be  opened  by  assigning  the  string  "waveaudio" 
as  follows: 

mciopen. lpstrDeviceType="waveaudio" 

Using  this  format  you  would  only  have  to  specify  MCI_OPEN_TYPE  for  dwflags. 


COMMENTS  :  Opening  the  driver  once  and  leaving  it  open  saves  time. 


You  can  then  use  this  driver  ID  in  the  following 

calls  to  open  elements  (wavefiles) .  This  saves  time  since 

if  you  open  an  element  without  specifying  am  open  device 

ID  mci  must  open  the  driver  and  close  the  driver  each 

time.  When  playing  multiple  elements  (wavefiles)  this 

excess  time  can  be  costly.  Therefore  to  conserve  time  you  should 

open  the  waveaudio  device,  play  the  waves,  close  the 

waveaudio  device . 

HISTORY  :  N/A 

WORD  FAR  PASCAL  OpenWaveDevice (void) 

{ 

MCI_DPEN_PARMS  mciopen; 

DWORD  dwRes; 

WORD  wGlobalDevicelD; 

DWORD  dwFlags; 

mciopen . wDeviceID=0 ; 

mciopen. lpstrDeviceType=(LPSTR)MCI_DEVTYPE_WAVEFORM_AUDIO; 

//  when  specifying  the  device  ID  you  must  also  include  the 
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//MCI_OPEN_TYPE  flag. 


dwFlags=  MCI_OPEN_TYPE_ID  |  MCI_OPEN_TYPE; 

dwRes=inciSendConiniand(0,MCI_OPEN,dwFlags,  (DWORD)  (LPSTR)&mciopen) ; 
if  (dwRes) 

{ 

ErrorProc(dwRes) ; 
return(FALSE) ; 

} 

wGlobalDeviceID=mciopen . wDevicelD ; 
return(wGlobalDevicelD) ; 

> 


FUNCTION  : 

;  OpenWaveFile(HWND,WORD,LPSTR) 

PURPOSE 

Open  wave  file  for  use  by  windows  program 

COMMENTS 

;  N/A 

HISTORY  ; 

;  N/A 

WORD  FAR  PASCAL  OpenWaveFile(wGlobalDeviceID,lpWaveName,lpAlias) 
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WORD  wGlobalDevicelD ; 


LPSTR  IpWaveName ; 

LPSTR  IpAlias; 

{ 

MCI_DPEN_PARMS  mciopen; 

//MCI_PLAY_PARMS  mciplay; 

DWORD  dwRes; 

WORD  wDevicelD; 

DWORD  dwFlags; 

//char  szWaveMessage [80] ; 
if  (! wGlobalDevicelD) 
return  FALSE; 

//  Open  the  wave  file  independent  of  the  device.  This  will  speed  up 
//  the  playing  of  the  wavefile. 

dwFlags=  MCI_OPEN_ELEMENT  |  MCI_0PEW_ALIAS; 
mciopen . lpstrElementName= (LPSTR) IpWaveName ; 

//  All  strings  must  be  initialized  otherwise  this  will  crash  under 
//  the  debug  version  of  mmsystem. 

mciopen. lpstrDeviceType="\0" ; 
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mciopeii . lpstrAlias= (LPSTR) IpAlias ; 


dwRes=mciSendCoiinnand(wGlobalDeviceID  ,MCI_OPEN,dwFlags ,  (DWORD)  (LPSTR) feme iopen) 

if  (dwRes) 

{ 

ErrorProc (dwRes) ; 
return  FALSE; 

} 


//  play  the  wave  file. 

wDeviceID=mciopen . wDevicelD ; 

return  wDevicelD; 

} 

FUNCTION  :  PlayWaveFile (wDevicelD) 

PURPOSE  :  This  fiuiction  takes  a  currently  open  device  ID  and  a 

string  representing  a  wave  file  and  plays  that  wavefile. 

When  the  wavefile  is  finished  playing  the  function  closes 
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the  wavefile. 


COMMENTS  :  dwFlags  is  set  to  MCI_WAIT  so  that  the  function  will  not 

return  until  the  wavefile  is  done  playing.  If  you  want 
the  function  to  return  before  the  wavefile  is  finished 
playing  omit  the  MCI_WAIT  flag. 

HISTORY  :  N/A 


***********  *!)<*******  Jtl****************************  *♦♦**♦*  ***********/ 

BOOL  FAR  PASCAL  PlayWaveFile (wDevicelD) 

WORD  wDevicelD; 

{ 

//MCI_OPEN_PARMS  mciopen; 

MCI_PLAY_PARMS  mciplay; 

DWORD  dwRes; 

DWORD  dwFlags; 

MSG  msg; 

//char  szWaveMessage [80] ; 

if  (! wDevicelD) 
return  FALSE; 
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//  Open  the  wave  file  independent  of  the  device.  This  will  speed  up 


//  the  playing  of  the  wavefile. 

mciplay .  dwFroni=0 ; 

dwFlags=MCI_FROM; 

dwRes=mciSendCommand(wDeviceID,MCI_PLAY,dwFlags, (DWORD) (LPSTR)&mciplay) ; 

if  (dwRes) 

{ 

ErrorProc (dwRes) ; 
return  FALSE; 

} 

while  (PeekMessage (&msg,HWindow,  NULL,  NULL,  PM_REM0VE)) 

{ 

TranslateMessage (&msg) ; 

DispatchMessage (&msg) ; 

> 

return  TRUE; 
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} 


FUNCTION  :  StopWaveFile(wDevicelD) 

PURPOSE  :  This  function  takes  a  currently  open  device  ID  and  a 

string  representing  a  wave  file  and  stops  play  of  that  wavefile. 
COMMENTS  :  N/A 
HISTORY  :  N/A 


BOOL  FAR  PASCAL  StopWaveFile (wDevicelD) 

WORD  wDevicelD; 

{ 

DWORD  dwRes; 

DWORD  dwFlags ; 
if  (! wDevicelD) 
return  FALSE; 

//  Open  the  wave  file  independent  of  the  device.  This  will  speed  up 
//  the  playing  of  the  wavefile. 

dwFlags=MCI_WAIT; 
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dwRes=mciSendCoiranand(wDeviceID,MCI_STOP,dwFlags, (DWORD) (LPVOID)NULL) ; 
if  (dwRes) 

{ 

ErrorProc(dwRes) ; 
return  FALSE; 

} 

return  TRUE; 

} 

FUNCTION  :  CloseWaveFile(HWND,WORD,LPSTR) 

PURPOSE  :  This  function  takes  a  currently  open  device  ID  and  a 

string  representing  a  wave  file  and  closes  that  wavefile. 

COMMENTS  :  N/A 

HISTORY  :  N/A 


BOOL  FAR  PASCAL  CloseWaveFile (wDevicelD) 
WORD  wDevicelD; 

{ 

DWORD  dwRes ; 
if  (! wDevicelD) 
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return  FALSE; 


//  Open  the  wave  file  independent  of  the  device.  This  will  speed  up 
//  the  playing  of  the  wavefile. 

dwRe  s =mc i S  endCoinmand (wDeviceID,MCI_CL0SE,0, NULL ) ; 
if  (dwRes) 

{ 

ErrorProc (dwRes) ; 
return  FALSE; 

} 

return  TRUE; 

} 

/****************  **********************=|C*************************>I<**** 

FUNCTION  :  CloseWaveDevice(HWND,WORD) 

PURPOSE  :  This  function  closes  a  currently  open  waveaudio  device. 

COMMENTS  :  To  conserve  system  resources  you  should  always  close  a 
device  when  you  are  not  using  it. 

HISTORY  :  N/A 
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BOOL  FAR  PASCAL  CloseWaveDevice(wGlobalDevicelD) 


WORD  wGlobalDevicelD; 

{ 


DWORD  dwRes; 
if  (! wGlobalDevicelD) 
return  FALSE; 


//  Close  the  global  device 


dwRes=mciSendComniand(wGlobalDeviceID,MCI_CLOSE,0,NULL) ; 


if  (dwRes) 


ErrorProc (dwRes) ; 
return(FALSE) ; 

} 

return (TRUE) ; 

> 

*******>!<  Drawing  Functions  **********************/ 
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FUNCTION 


void  Drawline (int  xl.int  yl,  int  x2,  int  y2) 


PURPOSE  :  This  function  draws  a  line  from  coord  (xl,yl)  to  (x2,y2): 

COMMENTS  :  N/A 

HISTORY  :  Written  by  Capt  Randy  Broussard 

void  DrawLine(int  xl.int  yl.int  x2,int  y2) 

{ 

HDC  hdc; 

hdc=GetDC(HWindow) ; 

SelectObject (hdc ,  GetStockObject (BLACK_PEN)) ; 

MoveTo(hdc ,xl , (int)yl) ; 

LineTo(hdc,x2,y2) ; 

ReleaseDC(HWindow,hdc) ; 

> 

FUNCTION  :  void  PrintText (int  x,int  y,char  *  String) 

PURPOSE  :  This  function  prints  a  text  string  beginning  at  coord  (x,y) 
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COMMENTS 


N/A 


HISTORY 


Written  by  Capt  Randy  Broussard 


void  PrintText(int  x,int  y.char  *  String) 

{ 

HOC  hdc; 


hdc=GetDC(HWindou) ; 

SetTextColor (hdc , Get Sy sColor ( COLOR.WINDOWTEXT) ) ; 

SetText Align (hdc, TA_LEFT  |  TA.TOP  |  TA_MOUPDATECP) ; 

TextOut (hdc , x , y , String , Istrlen (String) ) ; 

ReleaseDC(HWindow,hdc) ; 

> 

/  *********  *****iii******iltil<  ******************  *>!!****  :1c  It!  lie  initiate 


FUNCTION  :  void  EraseScreenO 

PURPOSE  :  This  function  erases  a  screen  by  painting  a  640x400  rectangle 
with  white 
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COMMENTS  :  N/A 


HISTORY  :  Written  by  Capt  Randy  Broussard 

*^!j|<****!(C!|C*!(C*j|C**>(tj(tj(ej|tJ|tS)C***j(tl|<**j(t**!(:***j(t=l'******************!(C***j|C!|c********/ 

void  EraseScreenO 

{ 

HOC  hdc; 

hdc=GetDC(HWindow) ; 

SelectObject (hdc .GetStockObject (WHITE.BRUSH) ) ; 

Rectangle (hdc ,0,0, 640 , 400) ; 

ReleaseDC(HWindow,hdc) ; 

} 

/ ********  jK************************************************************ 

FUNCTION  :  void  DrawBlackRectangle (int  xl,int  yl,int  x2,int  y2) 
PURPOSE  :  This  function  draws  a  rectangle  from  upper  left 
corner  at  (xl,yl)  to  lower  right  corner  at  (x2,y2) 
and  fills  it  in  with  black 
COMMENTS  :  N/A 
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HISTORY  :  Written  by  Capt  Randy  Broussard 


void  DrawBlackRe ct angle (int  xl,int  yl,int  x2,int  y2) 

{ 

HDC  hdc; 

hdc=GetDC(HWindow) ; 

SelectObject (hdc ,  GetStockObject (BLACK_PEM) ) ; 

SelectObject (hdc ,  GetStockObject (BLACK_BRUSH) ) ; 

Rectangle (hdc, xl,yl,x2,y2) ; 

ReleaseDC(HWindow,hdc) ; 

} 

FUNCTION  :  void  DrawWhiteRectangle (int  xl,int  yl.int  x2,int  y2) 
PURPOSE  :  This  function  draws  a  rectangle  from  upper  left 
corner  at  (xl,yl)  to  lower  right  corner  at  (x2,y2) 
and  fills  it  in  with  black 
COMMENTS  :  N/A 
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HISTORY  :  Written  by  Capt  Randy  Broussard 


void  DrawWhiteRectangle (int  xl.int  yl,int  x2,int  y2) 

{ 

HOC  hdc; 

hdc=GetDC(HWindow) ; 

SelectObj ect (hdc ,  GetStockObj ect (WHITE_PEN) ) ; 

SelectObject (hdc ,  GetStockObj ect (WHITE_BRUSH) ) ; 

Rectangle (hdc, xl,yl,x2,y2) ; 

ReleaseDC(HWindow,hdc) ; 

> 

^3((3)c3|c3fc:f(9|c:}(:(c9|c^^)|(’t<^’(<3((:(c3(c9|(:t(^^3(<3)<%3f(:^^]te3te9te:)c3)(3te%:|e](c](c^:(c9tc3|c3(c:le9{c:4c:tc:tc:fc:tc^e:tc9te9(c9{c9{c:(e9(c;tc:tc:4c:4c9tc)(cs|c:{c:{c:4c;tc 

FUNCTION  :  int  SilentDotsO 

PURPOSE  :  This  function,  when  called  from  main,  erases  the  screen  and  places 

a  black  cross-hair  in  the  center.  At  the  same  time,  a  call  to  the 
sound  card  is  made .  After  the  given  SoundDelayLoops  have  been 
executed,  a  black  dot  is  placed  on  either  the  left  or  right  side  of 
the  screen  -  depending  on  the  random  number  generated.  After 
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BetweenBoxLoops ,  a  second  black  dot  is  placed  on  the  opposite  side 


of  the  screen. 

COMMENTS  :  The  wavefile,  ’NoSound.wav’,  contains  only  static  -  no  audio  tone 

HISTORY  :  M/A 


int  SilentDotsO 

{ 

char  TempString[80] ; 

long  Loops .BetweenBoxLoops , SoundDelayLoops ; 

DWORD  Time l,Time2, Times ; 

EraseScreenO  ; 

DrawBlackRectangle (318 , 195 , 322 , 215) ; 

DrawBlackRectangle (310 , 203 , 330 , 207) ; 

Rx2=Rxl+Rwidth ; 

Ry2=Ryl+Rheight ; 

BetweenBoxLoops=(long)BetweenBoxTime*LoopsPerMilliSec/2 ; 
SomidDelayLoops= (long) SoundDelayTime*LoopsPerMilliSec/2 ; 
for (Loops=0 ; Loops<=5000000L ; Loops++) ; 
if (random(2)==l) 

{ 
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LRdir=l ; 


Nosound=l ; 

PlayWaveFile(wavelDN) ; 

f or (Loops=0 ; Loops<=SoundDelayLoops ;Loops++) ;  //delay  between  visual-audio  onset 
DrawBlackRectangle(Rxl ,Ryl jRx2,Ry2) ; 

f or (Loops=0 ; Loops<=BetweenBoxLoops ;Loops++) ;  //delay  between  visual  dot  display 
DrawBlackRe  ct angle (Rxl+dx+Rwidth , Ryl , Rx2+dx+Rwidth , Ry2) ; 

} 

else 

{ 

RLdir=l ; 

Nosound=l ; 

PlayWaveFile (wavelDN) ; 

for(Loops=0;Loops<=SoundDelayLoops;Loops++) ;  //delay  between  visual-audio  onset 
DrawBlackRectangle(Rxl+dx+Rwidth,Ryl ,Rx2+dx+Rwidth,Ry2) ; 

for(Loops=0;Loops<=BetweenBoxLoops;Loops++) ;  //delay  between  visual  dot  display 
DrawBlackRectangle (Rxl , Ry 1 , Rx2 , Ry2) ; 

> 

return  0; 

} 


FUNCTION  :  int  SyncDotsO 


PURPOSE  :  This  function,  when  called  from  main,  erases  the  screen  and  places 

a  black  cross-hair  in  the  center.  At  the  same  time,  a  call  to  the 
soiuid  card  is  made.  After  the  given  SoundDelayLoops  have  been 
executed,  a  black  dot  is  placed  on  either  the  left  or  right  side  of 
the  screen  -  depending  on  the  random  number  generated.  After 
BetweenBoxLoops ,  a  second  black  dot  is  placed  on  the  opposite  side 
of  the  screen. 

COMMENTS  :  The  visual  and  audio  stimuli  are  presented  in  order 

HISTORY  :  N/A 

int  SyncDotsO 

{ 

char  TempString  [80] ; 

long  Loops , BetweenBoxLoops , SoundDelayLoops ; 

//DWORD  Timel,Time2; 

EraseScreenO  ; 

DrawBlackRectangle (318 , 195 , 322 , 215) ; 

DrawBlackRectangle (310 , 203 , 330 , 207) ; 
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Rx2=Rxl+Rwidth; 


Ry2=Ryl+Rheight ; 

BetweenBoxLoops=(long)BetweenBoxTime*LoopsPerMilliSec/2 ; 
SoundDelayLoops=(long)SoundDelayTime*LoopsPerMilliSec/2 ; 
f or (Loops=0 ; Loops<=5000000L ; Loops++) ; 
if (random(2)==l) 

{ 

LRdir=l ; 

LRsouiid=l ; 

PlayWaveFile(wavelDL) ; 

f or (Loops=0 ; Loops<=SoundDelayLoops ;Loops++) ;  //delay  between  visual-audio  onset 
DrawBlackRectangle (Rxl , Ryl , Rx2 , Ry2) ; 

for(Loops=0;Loops<=BetweenBoxLoops;Loops++) ;  //delay  between  visual  dot  display 
DrawBlackRectangle (Rxl+dx+Rwidth, Ryl, Rx2+dx+Rwidth,Ry2) ; 

> 

else 

{ 

RLdir=l ; 

RLsound=l ; 

PlayWaveFile(wavelDH) ; 

for(Loops=0;Loops<=SoundDelayLoops;Loops++) ;  //delay  between  visual-audio  onset 
DrawBlackRectangle(Rxl+dx+Rwidth,Ryl,Rx2+dx+Rwidth,Ry2) ; 
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for(Loops=0;Loops<=BetweenBoxLoops ;Loops++) ;  //delay  between  visual  dot  display 
DrawBlackRectangle (Rxl , Ry 1 , Rx2 , Ry2) ; 

> 

& 

return  0; 

> 

FUNCTION  :  int  Outof SyncDotsO 

PURPOSE  :  This  function,  when  called  from  main,  erases  the  screen  and  places 

a  black  cross-hair  in  the  center.  At  the  same  time,  a  call  to  the 
soxmd  card  is  made .  After  the  given  SoundDelayLoops  have  been 
executed,  a  black  dot  is  placed  on  either  the  left  or  right  side  of 
the  screen  -  depending  on  the  random  number  generated.  After 
BetweenBoxLoops ,  a  second  black  dot  is  placed  on  the  opposite  side 
of  the  screen. 

COMMENTS  :  The  visual  and  audio  stimuli  are  presented  out-of-order  order 

HISTORY  :  N/A 

int  OutofSyncDots () 

{ 
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char  TempString [80] ; 

long  Loops , BoxOnLoops , BetweenBoxLoops , SoimdDelayLoops ; 

EraseScreenO  ; 

DrawBlackRect angle (318 , 195 , 322 , 215) ; 

DrawBlackRectangle (310 , 203 , 330 , 207) ; 

Rx2=Rxl+Rwidth ; 

Ry2=Ryl+Rheight ; 

BetweenBoxLoops=(long)BetweenBoxTime*LoopsPerMilliSec/2 ; 

SoundDelayLoops= (long) SoiindDelayTime*LoopsPerMilliSec/2 ; 
f or (Loops=0 ; Loops<=5000000L ; Loops++) ; 
if (random(2)==l) 

{ 

LRdir=l ; 

RLsound=l ; 

PlayWaveFile (waveIDH) ; 

for(Loops=0;Loops<=SoundDelayLoops;Loops++) ;  //delay  between  visual-audio  onset 
DrawBlackRectangle (Rxl , Ry 1 , Rx2 , Ry2) ; 

for(Loops=0;Loops<=BetweenBoxLoops;Loops++) ;  //delay  between  visual  dot  display 
DrawBlackRectangle (Rxl+dx+Rwidth,Ryl ,Rx2+dx+Rwidth,Ry2) ; 


RLdir=l ; 

LRsound=l ; 

PlayWaveFile(wavelDL) ; 

f or (Loops=0 ; Loops<=SoundDelayLoops ;Loops++) ;  //delay  between  visual-audio  onset 
DrawBlackRectangle(Rxl+dx+Rwidth,Ryl .Rx2+dx+Rwidth,Ry2) ; 

for(Loops=0;Loops<=BetweenBoxLoops ;Loops++) ;  //delay  between  visual  dot  display 
DrawBlackRectangle (Rxl ,Ryl ,Rx2 ,Ry2) ; 

} 

return  0; 

} 

/  ***  >|l****=(‘*>(<  *******♦♦*  J|!*****^<***>('*  ******  Jit********  *******  ************** 

FUNCTION  :  int  CalibrateTimer () 

PURPOSE  :  This  fvuiction  is  executed  at  the  beginning  of  the  program.  When 

called,  the  screen  is  erased  and  the  text  ’calibrating  timer  - 
please  wait’  placed  in  the  middle.  The  routing  then  makes  a  call 
to  the  windows  routine,  GetCurrentTime ,  to  get  the  current  windows 
time  in  milliseconds.  An  arbitrarily  large  number  of  iterative 
loops  are  then  executed,  after  which  the  current  windows  time,  in 
milliseconds,  is  again  retrieved.  Since  the  total  number  of  loops 
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executed  is  known,  as  well  as  the  starting  and  stopping  times  for 
execution  of  the  loops,  the  niimber  of  loops/millisecond  can  be 
calculated.  This  number  is  then  used  throughout  the  program  to 
determine  time  intervals. 

COMMENTS  :  N/A 

HISTORY  :  Written  by  Capt  Randy  Broussard,  modified  by  Capt  Don  Hill 

int  CalibrateTimer 0 

{ 

char  TempString[80] ; 
long  Loops; 

DWORD  Timel,Time2; 

EraseScreenO  ; 

PrintText (200 , 140, "Calibrating  Timer  -  Please  Wait"); 

Timel=GetCurrentTime 0 ; 
f or (Loops=0 ; Loops<=50000000L ; Loops++) ; 

Time2=GetCurrentTime 0 ; 

LoopsPerMilliSec=50000000L/ (Time2-Timel) ; 

wsprintf  (TempString,  "°/,ld  loops  performed  per  MilliSec"  ,LoopsPerMilliSec)  ; 
PrintText (200, 160, TempString) ; 
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PrintText (200, 180 , "Timer  Calibration  Completed"); 
randomize () ; 

DrawLine(320,200,320,210) ;  //draws  black  cross-hair 
DrawLine (315 , 205 , 325 , 205) ; 

return  0; 

} 

/lie*******************  Chan.ge  Parameters  Dialog  Box  ***********************/ 

FUNCTION  :  far  PASCAL  ChangeParametersMsgLoop(HWND  hDlg,  WORD  iMessage, 

WORD  wParam,  LONG  IParam) 

PURPOSE  :  This  Dialog  box,  created  using  Turbo  C++  Workshop,  allows  ascii 

values  to  be  entered  into  the  Change  Parameters  box  and  changed 
into  integers  for  use  throughout  the  program.  In  addition,  the 
routine  provides  the  user  with  an  up-to-date  look  at  various 
program  parameters 
COMMENTS  :  N/A 

HISTORY  :  Written  by  Capt  Randy  Broussard,  modified  by  Capt  Don  Hill 

******************  *********************,)C,|,,|,**,|,,|,,(,,|C*  *******  *************/ 
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#pragma  argsused 


BOOL  far  PASCAL  ChangeParametersMsgLoop(HWND  hDlg,  WORD  iMessage,  WORD  wParam,  LONG  IParam) 

{ 

char  TempString[80] ; 

switch(iMessage) 

{ 

case  WM.INITDIALOG: 
itoa(dx,TempString, 10) ; 

SetDlgltemText (hDlg, 101 , (LPSTR)TempString) ; 
itoa(Rwidth,TempString, 10) ; 

SetDlgltemText (hDlg, 102, (LPSTR)TempString) ; 
itoa(Rheight ,TempString, 10) ; 

SetDlgltemText (hDlg, 103, (LPSTR)TempString) ; 
itoa(BetweenBoxTime ,TempString, 10) ; 

SetDlgltemText (hDlg, 104, (LPSTR)TempString) ; 
itoa(SoimdDelayTime ,TempString, 10) ; 

SetDlgltemText (hDlg, 106, (LPSTR)TempString) ; 

return  TRUE; 
case  WM_COMMAND: 
switch(wParam) 

{ 
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case  IDOK: 


GetDlgltemText (hDlg, 101 , (LPSTR)TempString,sizeof (TempString) ) ; 
dx=atoi (TempString) ; 

GetDlgltemText (hDlg , 102 , (LPSTR) TempString , sizeof (TempString) ) ; 
Rwidth=atoi (TempString) ; 

GetDlgltemText (hDlg , 103 , (LPSTR) TempString , sizeof (TempString) ) ; 
Rheight=atoi (TempString) ; 

GetDlgltemText (hDlg, 104, (LPSTR) TempString, sizeof (TempString) ) ; 
BetweenBoxTime=atoi (TempString) ; 

GetDlgltemText (hDlg , 106 , (LPSTR) TempString , sizeof (TempString) ) ; 
SoundDelayTime=atoi (TempString) ; 

EndDialog(hDlg, 1) ; 
return  TRUE; 
case  WM.DESTROY: 

EndDialog(hDlg,0) ; 
return  TRUE; 

> 

break; 

} 

return  FALSE; 

> 

int  ChangeParametersO 
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{ 

static  FARPROC  IpfnDialogProc ; 
int  0kWasPressed=0; 

lpfnDialogProc=MakeProcInstaiice ( (FARPROC) ChangeParametersMsgLoop,ghInstance) ; 
OkWasPressed=DialogBox(ghInstance , "PARAMETERS" .HWindow, IpfnDialogProc) ; 
FreeProcInstance (IpfnDialogProc) ; 
return(OkWasPressed) ; 

} 

/di>icdc>)<«***«*«*******>i<>x  TestStatus  Dialog  Box  ******************/ 

FUNCTION  :  far  PASCAL  TestStatusMsgLoop(HWND  hDlg,  WORD  iMessage,  WORD 

wParam,  LONG  IParam) 

PURPOSE  :  This  Dialog  box,  created  using  Turbo  C++  Workshop,  allows  ascii 

values  to  be  entered  into  the  TestStatus  box  and  changed 
into  integers  for  use  throughout  the  program.  In  addition, 
this  program  provides  the  user  with  a  current  listing  of  all 
test  parameters 
COMMENTS  :  N/A 

HISTORY  :  Written  by  Capt  Don  Hill 


159 


*j|tj|<**j(cj(c*j|t******j(t******ji<j|tj|<j|ej|e*i(tj|c>(cj|c*j(!i|t**j(cj(cj(cj|t*!(c!|c**j|c!)t>(e******>|t*************/ 


#pragma  argsused 

BOOL  far  PASCAL  TestStatusMsgLoop(HWND  hDlg,  WORD  iMessage,  WORD  wParam,  LONG  IParam) 

{ 

char  TempString[80] ; 

switch (iMessage) 

{ 

case  WM.INITDIALOG; 
itoa(MovedRight jTempString, 10) ; 

SetDlgltemText (hDlg, 301, (LPSTR)TempString) ; 
itoa(MovedLeft ,TempString, 10) ; 

SetDlgltemText (hDlg, 302, (LPSTR)TempString) ; 
itoa(GuessedRight ,TempString, 10) ; 

SetDlgltemText (hDlg, 303, (LPSTR)TempString) ; 
itoa(GuessedLeft ,TempString, 10) ; 

SetDlgltemText (hDlg, 304, (LPSTR)TempString) ; 
itoa(Correct ,TempString, 10) ; 

SetDlgltemText (hDlg , 305 , (LPSTR) TempString) ; 
itoadncorrect  ,TempString,  10)  ; 

SetDlgltemText (hDlg, 306, (LPSTR) TempString) ; 
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itoa(TestCoiint  jTempString,  10)  ; 

SetDlgItemText(hDlg,307, (LPSTR)TempString) ; 
itoa(SilentCorrect .TempString, 10) ; 

SetDlgltemText (hDlg,308, (LPSTR)TempString) ; 
itoa(SilentIncorrect .TempString, 10) ; 

SetDlgltemText (hDlg, 309, (LPSTR)TempString) ; 
itoa(SyncCorrect  jTempString, 10) ; 

SetDlgltemText (hDlg, 310, (LPSTR)TempString) ; 
itoa(SyncIncorrect ,TempString, 10) ; 

SetDlgltemText (hDlg, 311 , (LPSTR)TempString) ; 
itoa(OutofSyncCorrect ,TempString, 10) ; 

SetDlgltemText (hDlg, 312, (LPSTR)TempString) ; 
itoa(0utof Syncincorrect ,TempString, 10) ; 

SetDlgltemText (hDlg, 313, (LPSTR)TempString) ; 

return  TRUE; 
case  WM_C0MMAND: 
switch(wParam) 

{ 

case  IDOK: 

GetDlgltemText (hDlg , 301 , (LPSTR) TempString , sizeof (TempStr ing) ) ; 
MovedRight=atoi(TempString) ; 

GetDlgltemText (hDlg, 302 , (LPSTR) TempStr ing, sizeof (TempString) ) ; 
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MovedLeft=atoi (TempString) ; 

GetDlgltemText (hDlg , 303 , (LPSTR) TempString , sizeof (TempString) ) ; 
GuessedRight=atoi (TempString) ; 

GetDlgltemText (hDlg , 304 , (LPSTR) TempString , sizeof (TempString) ) ; 
GuessedLeft=atoi (TempString) ; 

GetDlgltemText (hDlg , 305 , (LPSTR) TempString , sizeof (TempString) ) ; 
Correct=atoi (TempString) ; 

GetDlgltemText (hDlg , 306 , (LPSTR) TempString , sizeof (TempString) ) ; 
Incorrect=atoi (TempString) ; 

GetDlgltemText (hDlg, 307, (LPSTR) TempString, sizeof (TempString) ) ; 
TestCoiant=atoi  (TempString)  ; 

GetDlgltemText (hDlg , 308 , (LPSTR) TempString , sizeof (TempString) ) ; 
SilentCorrect=atoi (TempString) ; 

GetDlgltemText (hDlg , 309 , (LPSTR) TempString , sizeof (TempString) ) ; 
SilentIncorrect=atoi (TempString) ; 

GetDlgltemText (hDlg, 310, (LPSTR) TempString, sizeof (TempString) ) ; 
SyncCorrect=atoi (TempString) ; 

GetDlgltemText (hDlg, 311 , (LPSTR) TempString, sizeof (TempString) ) ; 
SyncIncorrect=atoi (TempString) ; 

GetDlgltemText (hDlg, 312 , (LPSTR) TempString, sizeof (TempString) ) ; 
Outof SyncCorrect=atoi (TempString) ; 

GetDlgltemText (hDlg , 313 , (LPSTR) TempString, sizeof (TempString) ) ; 
Outof SyncIncorrect=atoi (TempString) ; 
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EndDialogChDlg, 1) ; 
return  TRUE; 
case  WM_DESTROY: 

EndDialogChDlg, 0) ; 
return  TRUE; 

> 

break; 

} 

return  FALSE; 

> 

int  TestStatusO 

{ 

static  FARPROC  IpfnDialogProc ; 
int  0kWasPressed=0 ; 

lpfnDialogProc=MakeProcInstance ( (FARPROC) TestStatusMsgLoop,ghInstance) ; 
OkWasPressed=DialogBox(ghInstance , "STATUS" ,HWindow, IpfnDialogProc) ; 
FreeProcInstance (IpfnDialogProc) ; 
return(OkWasPressed) ; 

} 

/* *******!«<%* 4c lie* Main  Window  Message  proceedure  ****** itc*********** ★****/ 
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long  FAR  PASCAL  WndProc(HWND  hWnd,  unsigned  iMessage,  unsigned  wParam,  LONG  IParam) 

{ 

char  TempString[100] ; 

Rxl=  320-(dx/2)-Rwidth; 

switch (iMessage) 

{ 

case  WM_CQMMAND: 
s wit ch ( HI WORD ( IParam) ) 

{ 

case  WM_MENUSELECT: 
default : 
switch(wParam) 

{ 

case  101: 

ChangeParametersO  ; 

break; 

case  102: 

TestStatusO  ; 
break; 
case  103: 
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PostQuitMessage(O) ; 


break; 
case  203: 
GuessedLeft++; 
TestCount++; 
if  (LRdir==l) 

{ 

MovedRight++; 
Correct++; 
switch (rnum) 

{ 

case  0: 

SilentCorrect++ ; 

break; 

case  1: 

SyncCorrect++; 

break; 

case  2: 

Outof SyncCorrect++ ; 
break; 

> 

} 

else 
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■c 

MovedLeft++; 

Iiicorrect++; 
switch (rnum) 

{ 

case  0: 

SilentIncorrect++ ; 

break; 

case  1 : 

SyncIncorrect++ ; 

break; 

case  2: 

Outof SyncIncorrect++ ; 
break; 

> 

> 

RLdir=0 ; 

LRdir=0 ; 

RLsouiid=0 ; 

LRsound=0 ; 
if  (Test Count ==60) 

{ 

wsprintf (TempString,"THIS  CONCLUDES  THE  TEST"); 
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PriiitText(50,200,TempString) ; 


break; 

} 


for (Loops 1=0; Loops l<=2500000L;Loopsl++) ; 
wsprintf (TempString, "Ready") ; 

PrintText (50,380,TempString) ; 


break; 


case  204: 

GuessedRight++ ; 

TestCoiant++; 

if  (RLdir==l) 

{ 

MovedLeft++; 

Correct++; 

switch(rnim) 

{ 

case  0: 

SilentCorrect++ ; 

break; 

case  1: 

SyncCorrect++; 

break; 
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case  2: 


Outof SyncCorrect++ ; 
break; 

} 

} 

else 

{ 

MovedRight++ ; 
Incorrect++; 
switch (rnum) 

{ 

case  0: 

SilentIncorrect++ ; 

break; 

case  1 : 

SyncIncorrect++ ; 

break; 

case  2: 

Outof  SyncIiicorrect++ ; 
break; 

} 

} 

} 
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RLdir=0; 


LRdir=0 ; 

RLsound=0 ; 

LRsomid=0 ; 
if  (TestCount==60) 

{ 

wsprintf (TempString."THIS  CONCLUDES  THE  TEST"); 

PrintText (50 , 200 , TempString) ; 

break; 

} 

for (Loopsl=0 ; Loopsl<=2500000L ; Loopsl++) ; 
wsprintf (TempString, "Ready") ; 

PrintText (50, 380, TempString) ; 
break; 

case  205: 

//TestCount++; 

//randomize  0  ; 
flag  =  0; 
while (flag  ==  0) 

{ 

rnum  =  random(3) ; 

if((rnum  ==  0)  &&  (SilentCount  <  20)) 
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Si lent Count ++ ; 


SilentDotsO  ; 

flag=l; 

break; 

} 

else  if  ((rnum  ==  1)  &&  (SyncCount  <  20)) 

{ 

SyncCount++; 

SyncDotsO  ; 

flag=l; 

break; 

} 

else  if  ((rnum  ==  2)  &&  (Outof SyncCount  <  20)) 

{ 

Outof SyncCount++ ; 

Outof SyncDots 0 ; 

flag=l; 

break; 

> 

} 


break; 


MovedLeft=0; 


MovedRight=0 ; 

GuessedLeft=0; 

GuessedRight=0 ; 

Correct=0 ; 
lncorrect=0 ; 

RLdir=0 ; 

LRdir=0 ; 

RLsound=0 ; 

LRso\md=0 ; 

Nosoiind=0; 

SilentCorrect=0 ; 

SilentIncorrect=0 ; 

SyncCorrect=0 ; 

SyiicIncorrect=0 ; 

Outof SyncCorrect=0 ; 

Outof SyncIncorrect=0 ; 

TestCoimt=0 ; 

SilentCount=0 ; 

SyncCount=0; 

Outof SyncCoimt=0 ; 

EraseScreenO  ; 

wsprintf  (TempString,  "Test  Count:  ’/,d  "  ,TestCount)  ; 
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PrintText(50,380,TempString) ; 


break; 

} 

break; 

} 

break; 

case  WM.SIZE: 

if  (wParain==SIZEFULLSCREEN) 

{ 

ShowWindow(HWindow,  SW_SHOWNORMAL) ; 

} 

break; 
default : 

return  DefWindowProc(hWnd,  iMessage,  wParam,  IParam) ; 

} 

return(OL) ; 

} 

/jit*******************  Create  Main  Window  and  Buttons  *************/ 

#pragma  argsused 

int  PASCAL  WinMain (HANDLE  hinstance,  HANDLE  hPrevInstance , 


172 


LPSTR  IpszCmdLine ,  int  nCmdShow) 


{ 

HWND  hWnd; 

MSG  Msg; 

WNDCLASS  WndClass; 


ghlnstance=hlnstance ; 

if ( ! hPrevInstance) 

{ 

WndClass . style=CS_HREDRAW | CS_VREDRAW | CS_DBLCLKS ; 
WndClass . lpfnWndProc=WndProc ; 

WndClass .hicon  =  LoadIcon(ghInstance , "apmotion") ; 
WndClass .hbrBackground=GetStockObject (WHITE_BRUSH) ; 
WndClass .hCursor=NULL; 

WndClass  .hlnstance=hlnstcince ; 

WndClass . lpszMenuName="MAINMENU" ; 

WndClass .  lpszClassNaine="APMOTION" ; 

WndClass . cbClsExtra=0 ; 

WndClass . cbWndExtra=0 ; 

RegisterClass (feWndClass) ; 

} 
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hWnd=Cre  at  e Window ( 


"APMOTION", 

"Apparent  Motion  -  Noncausality  Test", 

WS.OVERLAPPED | WS_CAPTION I WS.SYSMENU | WS_MINIMIZEBDX | WS_MAXIMIZE , 

0. 

0, 

640, 

480, 

NULL, 

NULL, 

hinstance , 

NULL) ; 

HWindow=hWnd ; 

ShowWindow(hWnd,  nCmdShow) ; 

Button3=CreateWindow("BUTT0N" ,"GO",WS_CHILD|WS_VISIBLE|BS_PUSHBUTTON, 
315,410,75,25, HWindow , 205 , ghinstance , NULL) ; 

ShowWindow(Button3,  SW_SH0W) ; 


Button4=CreateWindow("BUTT0N","Left",WS_CHILD|WS_VISIBLE|BS_PUSHBUTT0N, 
480 ,410,75,25, HWindow , 203 , ghinstance , NULL) ; 
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ShowWindow(Button4,  SW_SHOW) ; 


Button5=CreateWindow( "BUTTON", "Right ",WS_CHILD I WS_VISIBLE I BS_PUSHBUTTON, 
560 ,410,75,25, HWindow , 204 , ghinstance , NULL) ; 

ShowWindow(Button5 ,  SW_SH0W) ; 

CalibrateTimer () ; 

//open  soimd  driver  and  .wav  files 

deviceID=OpenWaveDevice() ; 

waveIDH=OpenWaveFile (devicelD, (LPSTR) "rlwd.wav" , (LPSTR) "htone") ; 
waveIDL=OpenWaveFile(deviceID, (LPSTR) "lrwd.wav" , (LPSTR) "Itone") ; 
waveIDN=OpenWaveFile (devicelD , (LPSTR) "nosnd . wav" , (LPSTR) "notone " ) ; 
while (GetMessage(&Msg, NULL, 0,0)) 

{ 

TranslateMessage (&Msg) ; 

DispatchMessage (&Msg) ; 

} 

//close  .wav  files  and  sound  driver 

CloseWaveFile(wavelDL) ; 

CloseWaveFile(wavelDH) ; 

CloseWaveFile (wavelDN) ; 
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CloseWaveDevice (devicelD) ; 


return  Msg.wParam; 

} 
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G.2  *.RC  Code 


//  ObjectWindows  -  (C)  Copyright  1991  by  Borland  International 

#include  " windows. h" 

#include  "owlrc.h" 

#include  "inputdia.dlg" 

#include  "f iledial .dig" 

#include  " stdwnds . dig" 

#include  "editacc.rc" 

#include  "fileacc.rc" 

#include  "editmenu.rc" 

#include  "f ilemenu.rc" 

PARAMETERS  DIALOG  22,  15,  141,  152 

STYLE  DS_MODALFRAME  |  WS_P0PUP  |  WS_CAPTION  |  WS.SYSMENU 
CAPTION  "Parameters" 

{ 

EDITTEXT  101,  90,  12,  35,  12,  ES_LEFT  |  WS_CHILD  |  WS_VISIBLE  |  WS_B0RDER  |  WS_TABST0P 

EDITTEXT  104,  90,  32,  35,  12,  ES.LEFT  I  WS_CHILD  |  WS_VISIBLE  |  WS.BORDER  |  WS.TABSTOP 

RTEXT  "Motion  Step  Size",  -1,  21,  13,  65,  8 
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RTEXT  "Time  Between  Steps",  -1,  15,  32,  70,  8 


EDITTEXT  102,  90,  78,  35,  12,  ES.LEFT  |  WS_CHILD  |  WS_VISIBLE  |  WS.BORDER  |  WS.TABSTOP 

EDITTEXT  103,  90,  99,  35,  12,  ES_LEFT  |  WS_CHILD  |  WS_VISIBLE  |  WS_BORDER  I  WS.TABSTOP 

EDITTEXT  106,  90,  56,  35,  12,  ES.LEFT  |  WS.CHILD  I  WS_VISIBLE  |  WS.BORDER  I  WS.TABSTOP 

PUSHBUTTON  "&0k" ,  IDOK,  19,  121,  40,  20,  WS.CHILD  |  WS.VISIBLE  I  WS_TABSTOP 

PUSHBUTTON  "feCancel",  IDCANCEL,  87,  120,  40,  20,  WS_CHILD  |  WS_VISIBLE  |  WS.TABSTOP 
RTEXT  "Rectangle  Width",  -1,  19,  81,  65,  8 
RTEXT  "Rectangle  Height",  -1,  19,  102,  65,  8 
RTEXT  "Visual  Delay",  -1,  20,  58,  63,  8 

> 

STATUS  DIALOG  8,  -16,  198,  189 

STYLE  DS.MODALFRAME  |  WS.POPUP  |  WS.CAPTION  I  WS_SYSMENU 
CAPTION  "Status" 

{ 

RTEXT  "TEST  COUNT:",  -1,  2,  4,  65,  8,  SS_RIGHT  |  WS.CHILD  |  WS.VISIBLE  |  WS_GROUP 

EDITTEXT  307,  78,  2,  18,  12 

PUSHBUTTON  "&0K",  IDOK,  36,  166,  50,  14 

PUSHBUTTON  "feCancel",  IDCANCEL,  115,  166,  50,  14 

RTEXT  "INCORRECT:",  -1,  105,  71,  60,  8,  SS.RIGHT  |  WS_CHILD  |  WS_VISIBLE  |  WS_GROUP 

EDITTEXT  306,  175,  69,  18,  12 

EDITTEXT  302,  78,  18,  18,  12 

RTEXT  "STARTED  RIGHT:",  -1,  7,  20,  60,  8 
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EDITTEXT 

303, 

175,  18, 

18, 

12 

EDITTEXT 

304, 

175,  41, 

18, 

12 

RTEXT  "GUESSED  RIGHT:",  -1,  104,  21,  64,  8 
RTEXT  "GUESSED  LEFT:",  -1,  104,  44,  60,  8 
EDITTEXT  301,  78,  42,  18,  12 
RTEXT  "STARTED  LEFT:",  -1,  7,  44,  60,  8 
EDITTEXT  305,  78,  69,  18,  12 
RTEXT  "CORRECT:",  -1,  8,  72,  60,  8 

//CONTROL  -1,  "BorShade",  BSS.HDIP  I  BSS.LEFT  |  WS_CHILD  I  WS.VISIBLE,  0,  60,  198,  10 

EDITTEXT  308,  78,  105,  18,  12 

RTEXT  "silent:",  -1,  8,  108,  60,  8 

EDITTEXT  310,  78,  122,  18,  12 

RTEXT  "so\md-in-sync : " ,  -1,  8,  125,  60,  8 

EDITTEXT  312,  78,  139,  18,  12 

RTEXT  "sound-out-of-sync : " ,  -1,  8,  142,  60,  8 

RTEXT  "By  Test  Mode:",  -1,  8,  90,  60,  8 

EDITTEXT  309,  175,  104,  18,  12 

RTEXT  "silent:",  -1,  105,  107,  60,  8 

EDITTEXT  311,  175,  121,  18,  12 

RTEXT  "sound-in-sync:",  -1,  105,  124,  60,  8 

EDITTEXT  313,  175,  138,  18,  12 

RTEXT  "sound-out-of-sync : " ,  -1,  105,  141,  60,  8 
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MAINMENU  MENU 


{ 

POPUP  "&Graph" 

{ 

MENUITEM  "feChange  Parameters",  101 
MENUITEM  "&Test  Status",  102 
MENUITEM  "&Reset  Parameters",  105 
MENUITEM  "E&xit",  103 

> 

> 

apmotion  ICON 

{ 

’00  00  01  00  01  00  20  20  10  00  00  00  00  00  E8  02’ 

’00  00  16  00  00  00  28  00  00  00  20  00  00  00  40  00’ 

’00  00  01  00  04  00  00  00  00  00  80  02  00  00  00  00’ 

’00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00’ 

’00  00  00  00  80  00  00  80  00  00  00  80  80  00  80  00’ 

’00  00  80  00  80  00  80  80  00  00  CO  CO  CO  00  80  80’ 

’80  00  00  00  FF  00  00  FF  00  00  00  FF  FF  00  FF  00’ 
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’00  00  FF  00  FF  00  FF  FF 
’99  99  99  99  99  99  99  99 
’FF  FF  FF  FF  FF  FF  FF  FF 
’FF  FF  FF  FF  FF  FF  FF  FF 
’FF  FF  FF  FF  FF  FF  FF  FF 
’FF  FF  FF  FF  FF  FF  FF  FF 
’FF  FF  FF  FF  FF  FF  FF  FF 
’FF  FF  FF  FF  FF  FF  FF  FF 
’FF  FF  FF  FF  FF  FF  FF  FF 
’FF  FF  FF  FF  FF  FF  FF  FF 
’FF  FF  FF  FF  FF  FF  FF  FF 
’FF  FF  FF  FF  FF  FF  FF  FF 
’FF  FF  FF  FF  FF  FF  FF  FF 
’FF  FF  FF  FF  FF  FF  FF  FO 
’FF  FF  FF  FF  FF  FF  FF  FF 
’FF  FF  FF  FF  FF  FF  00  FF 
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